[Git][debian-gis-team/pyosmium][buster-backports] 21 commits: Update lintian overrides.
Bas Couwenberg
gitlab at salsa.debian.org
Tue Jan 19 04:55:56 GMT 2021
Bas Couwenberg pushed to branch buster-backports at Debian GIS Project / pyosmium
Commits:
3a1bcc18 by Bas Couwenberg at 2020-10-30T12:18:08+01:00
Update lintian overrides.
- - - - -
52e3ad3e by Bas Couwenberg at 2020-10-30T12:31:18+01:00
Add patch to fix FTFBS with pybind11 2.6.0.
- - - - -
2e98bf2b by Bas Couwenberg at 2020-10-30T12:42:56+01:00
Set distribution to unstable.
- - - - -
6bf05da7 by Bas Couwenberg at 2020-11-06T19:49:40+01:00
Bump watch file version to 4.
- - - - -
c105af43 by Bas Couwenberg at 2020-11-12T04:07:30+01:00
New upstream version 3.1.0
- - - - -
78fdb689 by Bas Couwenberg at 2020-11-12T04:07:34+01:00
Update upstream source from tag 'upstream/3.1.0'
Update to upstream version '3.1.0'
with Debian dir 852a8e104050ab76aeae604bcbe202d22a27c693
- - - - -
cb887db3 by Bas Couwenberg at 2020-11-12T04:09:45+01:00
New upstream release.
- - - - -
bb2270fe by Bas Couwenberg at 2020-11-12T04:11:50+01:00
Drop pybind11-2.6.0.patch, fixed upstream.
- - - - -
91619a0b by Bas Couwenberg at 2020-11-12T04:11:59+01:00
Set distribution to unstable.
- - - - -
56f60981 by Bas Couwenberg at 2020-11-28T14:12:55+01:00
Bump Standards-Version to 4.5.1, no changes.
- - - - -
fe39e491 by Bas Couwenberg at 2021-01-12T17:29:16+01:00
New upstream version 3.1.1
- - - - -
8f043c58 by Bas Couwenberg at 2021-01-12T17:29:21+01:00
Update upstream source from tag 'upstream/3.1.1'
Update to upstream version '3.1.1'
with Debian dir 2b85d5bae118e576bf386fde0e5df28c54657355
- - - - -
205ced86 by Bas Couwenberg at 2021-01-12T17:29:36+01:00
New upstream release.
- - - - -
85604710 by Bas Couwenberg at 2021-01-12T17:31:01+01:00
Bump minimum required libosmium2-dev to 2.16.0.
- - - - -
871c4837 by Bas Couwenberg at 2021-01-12T17:31:13+01:00
Set distribution to unstable.
- - - - -
f91ca28a by Bas Couwenberg at 2021-01-14T05:35:15+01:00
New upstream version 3.1.2
- - - - -
ba3d9201 by Bas Couwenberg at 2021-01-14T05:35:19+01:00
Update upstream source from tag 'upstream/3.1.2'
Update to upstream version '3.1.2'
with Debian dir afbef81feeccda956f255708fba925be89dea2f8
- - - - -
bb76a749 by Bas Couwenberg at 2021-01-14T05:37:39+01:00
New upstream release.
- - - - -
30b0de9d by Bas Couwenberg at 2021-01-14T05:38:07+01:00
Set distribution to unstable.
- - - - -
0d508b3a by Bas Couwenberg at 2021-01-19T05:43:07+01:00
Merge tag 'debian/3.1.2-1' into buster-backports
releasing package pyosmium version 3.1.2-1
- - - - -
8a74dca2 by Bas Couwenberg at 2021-01-19T05:43:13+01:00
Rebuild for buster-backports.
- - - - -
28 changed files:
- + .github/actions/install-dependencies/action.yml
- + .github/workflows/ci.yml
- − .travis.yml
- CHANGELOG.md
- CMakeLists.txt
- README.md
- + cmake/FindLZ4.cmake
- cmake/FindOsmium.cmake
- debian/changelog
- debian/control
- − debian/source/lintian-overrides
- debian/watch
- doc/updating_osm_data.rst
- examples/amenity_list.py
- examples/pub_names.py
- examples/use_nodecache.py
- lib/io.cc
- lib/merge_input_reader.cc
- setup.py
- src/osmium/helper.py
- src/osmium/osm/__init__.py
- src/osmium/osm/mutable.py
- src/osmium/replication/server.py
- src/osmium/replication/utils.py
- src/osmium/version.py
- test/test_replication.py
- tools/pyosmium-get-changes
- tools/pyosmium-up-to-date
Changes:
=====================================
.github/actions/install-dependencies/action.yml
=====================================
@@ -0,0 +1,18 @@
+name: 'Install pyosmium dependencies'
+
+runs:
+ using: "composite"
+
+ steps:
+ - name: Install pip dependencies
+ run: |
+ python -m pip install --upgrade pip
+ pip install nose shapely setuptools
+ shell: bash
+
+ - name: Install package dependencies
+ run: |
+ git clone --quiet --depth 1 https://github.com/osmcode/libosmium.git contrib/libosmium
+ git clone --quiet --depth 1 https://github.com/mapbox/protozero.git contrib/protozero
+ git clone --quiet --depth 1 https://github.com/pybind/pybind11.git contrib/pybind11
+ shell: bash
=====================================
.github/workflows/ci.yml
=====================================
@@ -0,0 +1,141 @@
+name: CI
+
+on: [ push, pull_request ]
+
+jobs:
+ build-default:
+ runs-on: ubuntu-20.04
+
+ strategy:
+ matrix:
+ python-version: [3.6, 3.7, 3.8]
+
+ steps:
+ - uses: actions/checkout at v2
+
+ - name: Set up Python ${{ matrix.python-version }}
+ uses: actions/setup-python at v2
+ with:
+ python-version: ${{ matrix.python-version }}
+
+ - name: Install packages
+ run: sudo apt-get install -y -qq libboost-dev libexpat1-dev zlib1g-dev libbz2-dev libproj-dev libgeos-dev liblz4-dev
+
+ - uses: ./.github/actions/install-dependencies
+
+ - name: Build package
+ run: python setup.py build
+ shell: bash
+
+ - name: Run tests
+ run: python run_tests.py
+ shell: bash
+ working-directory: test
+
+
+ build-ubuntu-1604:
+ runs-on: ubuntu-16.04
+
+ strategy:
+ matrix:
+ compiler: [gcc, clang]
+
+ steps:
+ - uses: actions/checkout at v2
+
+ - uses: actions/setup-python at v2
+ with:
+ python-version: 3.5
+
+ - name: Install packages
+ run: sudo apt-get install -y -qq libboost-dev libexpat1-dev zlib1g-dev libbz2-dev libproj-dev libgeos-dev liblz4-dev
+
+ - uses: ./.github/actions/install-dependencies
+
+ - name: Build package
+ run: python setup.py build
+ shell: bash
+ env:
+ CC: gcc-5
+ CXX: g++-5
+ if: ${{ matrix.compiler == 'gcc' }}
+
+ - name: Build package
+ run: python setup.py build
+ shell: bash
+ env:
+ CC: clang-6.0
+ CXX: clang++-6.0
+ if: ${{ matrix.compiler == 'clang' }}
+
+ - name: Run tests
+ run: python run_tests.py
+ shell: bash
+ working-directory: test
+
+ build-ubuntu-2004:
+ runs-on: ubuntu-20.04
+
+ strategy:
+ matrix:
+ compiler: [gcc, clang]
+
+ steps:
+ - uses: actions/checkout at v2
+
+ - uses: actions/setup-python at v2
+ with:
+ python-version: 3.9
+
+ - name: Install packages
+ run: sudo apt-get install -y -qq libboost-dev libexpat1-dev zlib1g-dev libbz2-dev libproj-dev libgeos-dev liblz4-dev
+
+ - uses: ./.github/actions/install-dependencies
+
+ - name: Build package
+ run: python setup.py build
+ shell: bash
+ env:
+ CC: gcc-10
+ CXX: g++-10
+ if: ${{ matrix.compiler == 'gcc' }}
+
+ - name: Build package
+ run: python setup.py build
+ shell: bash
+ env:
+ CC: clang-10
+ CXX: clang++-10
+ if: ${{ matrix.compiler == 'clang' }}
+
+ - name: Run tests
+ run: python run_tests.py
+ shell: bash
+ working-directory: test
+
+ build-macos:
+ runs-on: macos-latest
+
+ steps:
+ - uses: actions/checkout at v2
+
+ - uses: actions/setup-python at v2
+ with:
+ python-version: 3
+
+ - name: Install packages
+ run: brew install boost geos lz4
+ shell: bash
+
+ - uses: ./.github/actions/install-dependencies
+
+ - name: Build package
+ run: python setup.py build
+ shell: bash
+
+ - name: Run tests
+ run: python run_tests.py
+ shell: bash
+ working-directory: test
+
+
=====================================
.travis.yml deleted
=====================================
@@ -1,84 +0,0 @@
-#-----------------------------------------------------------------------------
-#
-# Configuration for continuous integration service at travis-ci.org
-#
-#-----------------------------------------------------------------------------
-
-
-matrix:
- include:
- - os: linux
- compiler: "clang-3.4"
- dist: xenial
- language: cpp
- - os: linux
- compiler: "clang-7"
- dist: bionic
- language: cpp
- - os: linux
- compiler: "gcc-5"
- dist: xenial
- language: cpp
- - os: linux
- compiler: "gcc-8"
- dist: bionic
- language: cpp
- - os: linux
- compiler: gcc
- python: 3.4
- dist: xenial
- language: python
- - os: linux
- compiler: gcc
- python: 3.5
- dist: bionic
- language: python
- - os: linux
- compiler: gcc
- python: 3.6
- dist: bionic
- language: python
- - os: linux
- compiler: gcc
- python: 3.7
- dist: bionic
- language: python
- - os: linux
- compiler: gcc
- python: 3.8
- dist: bionic
- language: python
- - os: osx
- osx_image: xcode7
- compiler: clang
- language: cpp
- - os: osx
- osx_image: xcode10.1
- compiler: clang
- language: cpp
-
-# http://docs.travis-ci.com/user/apt/
-addons:
- apt:
- packages:
- - libboost-dev
- - python3
- - python3-dev
- - python3-pip
-
-install:
- - git clone --quiet --depth 1 https://github.com/osmcode/libosmium.git contrib/libosmium
- - git clone --quiet --depth 1 https://github.com/mapbox/protozero.git contrib/protozero
- - git clone --quiet --depth 1 https://github.com/pybind/pybind11.git contrib/pybind11
- - if [ "$TRAVIS_OS_NAME" = 'osx' ]; then
- pip3 install --user -q nose mock shapely setuptools;
- else
- pip3 install -q nose mock shapely setuptools;
- fi
-
-script:
- - python3 --version
- - python3 setup.py build
- - cd test
- - python3 run_tests.py
-
=====================================
CHANGELOG.md
=====================================
@@ -4,6 +4,37 @@
All notable changes to this project will be documented in this file.
This project adheres to [Semantic Versioning](http://semver.org/).
+## [3.1.2] - 2021-01-13
+
+### Fixed
+
+- package pypi source wheel with correct libosmium version
+
+## [3.1.1] - 2021-01-12
+
+### Added
+
+- support for lz4 compression (compled in when library is found)
+
+### Changed
+
+- update to libosmium 2.16.0
+- update to pybind11 2.6.1
+
+## [3.1.0] - 2020-11-03
+
+### Added
+
+### Changed
+
+- improved help messages for update tools
+- update to pybind11 2.6
+
+### Fixed
+
+- pyosmium-up-to-date: fix missing flag when applying changes to history files
+- smaller linting and documentation issues
+
## [3.0.1] - 2020-07-25
### Added
=====================================
CMakeLists.txt
=====================================
@@ -1,18 +1,48 @@
cmake_minimum_required(VERSION 2.8.12)
project(pyosmium)
+set(PACKAGE_VERSION 3.0.1)
+
+option(WITH_LZ4 "Build with lz4 support for PBF files" ON)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
-find_package(Osmium 2.15 REQUIRED COMPONENTS io pbf xml)
+find_package(Osmium 2.16 REQUIRED COMPONENTS io pbf xml)
+
+if(WITH_LZ4)
+find_package(LZ4)
+
+ if(LZ4_FOUND)
+ message(STATUS "lz4 library found, compiling with it")
+ add_definitions(-DOSMIUM_WITH_LZ4)
+ include_directories(SYSTEM ${LZ4_INCLUDE_DIRS})
+ list(APPEND OSMIUM_LIBRARIES ${LZ4_LIBRARIES})
+ else()
+ message(WARNING "lz4 library not found, compiling without it")
+ endif()
+else()
+ message(STATUS "Building without lz4 support: Set WITH_LZ4=ON to change this")
+endif()
+
include_directories(SYSTEM ${OSMIUM_INCLUDE_DIRS} ${PROTOZERO_INCLUDE_DIR})
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/lib)
+if(NOT "${CMAKE_CXX_STANDARD}")
+ if(MSVC)
+ set(CMAKE_CXX_STANDARD 14)
+ else()
+ set(CMAKE_CXX_STANDARD 11)
+ endif()
+endif()
+
+# required for pybind11 < 2.6
if(MSVC)
- set(PYBIND11_CPP_STANDARD /std:c++14)
+ set(PYBIND11_CPP_STANDARD /std=c++${CMAKE_CXX_STANDARD})
else()
- set(PYBIND11_CPP_STANDARD -std=c++11)
+ set(PYBIND11_CPP_STANDARD -std=c++${CMAKE_CXX_STANDARD})
endif()
+message(STATUS "Building in C++${CMAKE_CXX_STANDARD} mode")
+
if(PYBIND11_PREFIX)
add_subdirectory(${PYBIND11_PREFIX} contrib/pybind11)
else()
=====================================
README.md
=====================================
@@ -4,7 +4,7 @@ Provides Python bindings for the [Libosmium](https://github.com/osmcode/libosmiu
library, a library for working with OpenStreetMap data in a fast and flexible
manner.
-[](http://travis-ci.org/osmcode/pyosmium)
+[](https://github.com/osmcode/pyosmium/actions?query=workflow%3ACI)
[](https://ci.appveyor.com/project/lonvia/pyosmium/branch/master)
## Installation
@@ -34,7 +34,7 @@ command installs all required packages:
pyosmium has the following dependencies:
- * [libosmium](https://github.com/osmcode/libosmium)
+ * [libosmium](https://github.com/osmcode/libosmium) >= 2.16.0
* [protozero](https://github.com/mapbox/protozero)
* [cmake](https://cmake.org/)
* [Pybind11](https://github.com/pybind/pybind11) >= 2.2
=====================================
cmake/FindLZ4.cmake
=====================================
@@ -0,0 +1,38 @@
+find_path(LZ4_INCLUDE_DIR
+ NAMES lz4.h
+ DOC "lz4 include directory")
+mark_as_advanced(LZ4_INCLUDE_DIR)
+find_library(LZ4_LIBRARY
+ NAMES lz4 liblz4
+ DOC "lz4 library")
+mark_as_advanced(LZ4_LIBRARY)
+
+if (LZ4_INCLUDE_DIR)
+ file(STRINGS "${LZ4_INCLUDE_DIR}/lz4.h" _lz4_version_lines
+ REGEX "#define[ \t]+LZ4_VERSION_(MAJOR|MINOR|RELEASE)")
+ string(REGEX REPLACE ".*LZ4_VERSION_MAJOR *\([0-9]*\).*" "\\1" _lz4_version_major "${_lz4_version_lines}")
+ string(REGEX REPLACE ".*LZ4_VERSION_MINOR *\([0-9]*\).*" "\\1" _lz4_version_minor "${_lz4_version_lines}")
+ string(REGEX REPLACE ".*LZ4_VERSION_RELEASE *\([0-9]*\).*" "\\1" _lz4_version_release "${_lz4_version_lines}")
+ set(LZ4_VERSION "${_lz4_version_major}.${_lz4_version_minor}.${_lz4_version_release}")
+ unset(_lz4_version_major)
+ unset(_lz4_version_minor)
+ unset(_lz4_version_release)
+ unset(_lz4_version_lines)
+endif ()
+
+include(FindPackageHandleStandardArgs)
+find_package_handle_standard_args(LZ4
+ REQUIRED_VARS LZ4_LIBRARY LZ4_INCLUDE_DIR
+ VERSION_VAR LZ4_VERSION)
+
+if (LZ4_FOUND)
+ set(LZ4_INCLUDE_DIRS "${LZ4_INCLUDE_DIR}")
+ set(LZ4_LIBRARIES "${LZ4_LIBRARY}")
+
+ if (NOT TARGET LZ4::LZ4)
+ add_library(LZ4::LZ4 UNKNOWN IMPORTED)
+ set_target_properties(LZ4::LZ4 PROPERTIES
+ IMPORTED_LOCATION "${LZ4_LIBRARY}"
+ INTERFACE_INCLUDE_DIRECTORIES "${LZ4_INCLUDE_DIR}")
+ endif ()
+endif ()
=====================================
cmake/FindOsmium.cmake
=====================================
@@ -34,6 +34,7 @@
# gdal - include if you want to use any of the OGR functions
# proj - include if you want to use any of the Proj.4 functions
# sparsehash - include if you use the sparsehash index
+# lz4 - include support for LZ4 compression of PBF files
#
# You can check for success with something like this:
#
@@ -116,14 +117,21 @@ if(Osmium_USE_PBF)
find_package(Threads)
find_package(Protozero 1.6.3)
+ if(Osmium_USE_LZ4)
+ find_package(LZ4 REQUIRED)
+ add_definitions(-DOSMIUM_WITH_LZ4)
+ endif()
+
list(APPEND OSMIUM_EXTRA_FIND_VARS ZLIB_FOUND Threads_FOUND PROTOZERO_INCLUDE_DIR)
if(ZLIB_FOUND AND Threads_FOUND AND PROTOZERO_FOUND)
list(APPEND OSMIUM_PBF_LIBRARIES
${ZLIB_LIBRARIES}
+ ${LZ4_LIBRARIES}
${CMAKE_THREAD_LIBS_INIT}
)
list(APPEND OSMIUM_INCLUDE_DIRS
${ZLIB_INCLUDE_DIR}
+ ${LZ4_INCLUDE_DIRS}
${PROTOZERO_INCLUDE_DIR}
)
else()
=====================================
debian/changelog
=====================================
@@ -1,3 +1,38 @@
+pyosmium (3.1.2-1~bpo10+1) buster-backports; urgency=medium
+
+ * Rebuild for buster-backports.
+
+ -- Bas Couwenberg <sebastic at debian.org> Tue, 19 Jan 2021 05:43:10 +0100
+
+pyosmium (3.1.2-1) unstable; urgency=medium
+
+ * New upstream release.
+
+ -- Bas Couwenberg <sebastic at debian.org> Thu, 14 Jan 2021 05:37:57 +0100
+
+pyosmium (3.1.1-1) unstable; urgency=medium
+
+ * New upstream release.
+ * Bump Standards-Version to 4.5.1, no changes.
+ * Bump minimum required libosmium2-dev to 2.16.0.
+
+ -- Bas Couwenberg <sebastic at debian.org> Tue, 12 Jan 2021 17:31:04 +0100
+
+pyosmium (3.1.0-1) unstable; urgency=medium
+
+ * New upstream release.
+ * Bump watch file version to 4.
+ * Drop pybind11-2.6.0.patch, fixed upstream.
+
+ -- Bas Couwenberg <sebastic at debian.org> Thu, 12 Nov 2020 04:11:51 +0100
+
+pyosmium (3.0.1-2) unstable; urgency=medium
+
+ * Update lintian overrides.
+ * Add patch to fix FTFBS with pybind11 2.6.0.
+
+ -- Bas Couwenberg <sebastic at debian.org> Fri, 30 Oct 2020 12:42:42 +0100
+
pyosmium (3.0.1-1~bpo10+1) buster-backports; urgency=medium
* Rebuild for buster-backports.
=====================================
debian/control
=====================================
@@ -11,7 +11,7 @@ Build-Depends: cmake (>= 2.8.12),
libexpat1-dev,
libgdal-dev,
libgeos++-dev,
- libosmium2-dev (>= 2.15.6),
+ libosmium2-dev (>= 2.16.0),
libsparsehash-dev,
pybind11-dev,
python3-all-dev,
@@ -22,7 +22,7 @@ Build-Depends: cmake (>= 2.8.12),
Build-Depends-Indep:
python3-sphinx,
python3-sphinxcontrib.autoprogram
-Standards-Version: 4.5.0
+Standards-Version: 4.5.1
Vcs-Browser: https://salsa.debian.org/debian-gis-team/pyosmium/
Vcs-Git: https://salsa.debian.org/debian-gis-team/pyosmium.git -b buster-backports
Homepage: https://osmcode.org/pyosmium/
=====================================
debian/source/lintian-overrides deleted
=====================================
@@ -1,3 +0,0 @@
-# Not worth the effort
-testsuite-autopkgtest-missing
-
=====================================
debian/watch
=====================================
@@ -1,4 +1,4 @@
-version=3
+version=4
opts=\
dversionmangle=s/\+(debian|dfsg|ds|deb)\d*$//,\
uversionmangle=s/(\d)[_\.\-\+]?((RC|rc|pre|dev|beta|alpha)\d*)$/$1~$2/,\
=====================================
doc/updating_osm_data.rst
=====================================
@@ -33,14 +33,15 @@ There are multiple sources for OSM change files available:
for planet-wide updates. There are change files for
minutely, hourly and daily intervals available.
- * `Geofabrik <http://download.geofabrik.de>`_ offers daily change files
+ * `Geofabrik <https://download.geofabrik.de>`_ offers daily change files
for all its updates. See the extract page for a link to the replication URL.
Note that change files go only about 3 months back. Older files are deleted.
- * `openstreetmap.fr <http://download.geofabrik.de>`_ offers minutely change
- files for all its extracts.
+ * download.openstreetmap.fr offers
+ `minutely change files <https://download.openstreetmap.fr/replication/>`_
+ for all its `extracts <https://download.openstreetmap.fr/extracts/>`_..
-For other services also check out the list of services on the
+For other services also check out the list of providers on the
`OSM wiki <https://wiki.openstreetmap.org/wiki/Planet.osm>`_.
Updating a planet or extract
=====================================
examples/amenity_list.py
=====================================
@@ -13,7 +13,7 @@ wkbfab = o.geom.WKBFactory()
class AmenityListHandler(o.SimpleHandler):
- def print_amenity(amenity, tags, lon, lat):
+ def print_amenity(self, tags, lon, lat):
name = tags.get('name', '')
print("%f %f %-15s %s" % (lon, lat, tags['amenity'], name))
=====================================
examples/pub_names.py
=====================================
@@ -7,9 +7,8 @@ import sys
class NamesHandler(osmium.SimpleHandler):
def output_pubs(self, tags):
- if tags.get('amenity') == 'pub':
- if 'name' in tags:
- print(tags['name'])
+ if tags.get('amenity') == 'pub' and 'name' in tags:
+ print(tags['name'])
def node(self, n):
self.output_pubs(n.tags)
=====================================
examples/use_nodecache.py
=====================================
@@ -9,9 +9,8 @@ class WayHandler(o.SimpleHandler):
def way(self, w):
for n in w.nodes:
- n.lat, n.lon # throws an exception if the coordinates are missing
loc = idx.get(n.ref)
- print("%d %s" %(w.id, len(w.nodes)))
+ print("%d %s" % (w.id, len(w.nodes)))
if len(sys.argv) != 3:
print("Usage: python create_nodecache.py <osm file> <node cache>")
=====================================
lib/io.cc
=====================================
@@ -59,7 +59,7 @@ PYBIND11_MODULE(io, m)
py::class_<osmium::io::Writer>(m, "Writer",
"Class for writing OSM data to a file. This class just encapsulates an "
- "OSM file,. Have a look `osmium.SimpleWriter` for a high-level interface "
+ "OSM file. Have a look `osmium.SimpleWriter` for a high-level interface "
"for writing out data.")
.def(py::init<std::string>())
.def(py::init<osmium::io::File>())
=====================================
lib/merge_input_reader.cc
=====================================
@@ -53,8 +53,7 @@ namespace {
class MergeInputReader
{
public:
- void apply(BaseHandler& handler, std::string const &idx = "",
- bool simplify = true)
+ void apply(BaseHandler& handler, std::string const &idx, bool simplify)
{
if (idx.empty())
apply_without_location(handler, simplify);
@@ -63,7 +62,7 @@ public:
}
void apply_to_reader(osmium::io::Reader &reader, osmium::io::Writer &writer,
- bool with_history = true)
+ bool with_history)
{
auto input = osmium::io::make_input_iterator_range<osmium::OSMObject>(reader);
if (with_history) {
@@ -110,7 +109,7 @@ public:
}
private:
- void apply_without_location(BaseHandler& handler, bool simplify = true)
+ void apply_without_location(BaseHandler& handler, bool simplify)
{
if (simplify) {
objects.sort(osmium::object_order_type_id_reverse_version());
@@ -133,7 +132,7 @@ private:
}
void apply_with_location(BaseHandler& handler, std::string const &idx,
- bool simplify = true)
+ bool simplify)
{
using Index_fab =
osmium::index::MapFactory<osmium::unsigned_object_id_type, osmium::Location>;
=====================================
setup.py
=====================================
@@ -105,6 +105,9 @@ class CMakeBuild(build_ext):
if 'BOOST_PREFIX' in env:
cmake_args += ['-DBOOST_ROOT={}'.format(env['BOOST_PREFIX'])]
+ if 'CMAKE_CXX_STANDARD' in env:
+ cmake_args += ['-DCMAKE_CXX_STANDARD={}'.format(env['CMAKE_CXX_STANDARD'])]
+
if not os.path.exists(self.build_temp):
os.makedirs(self.build_temp)
subprocess.check_call(['cmake', ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env)
@@ -112,8 +115,8 @@ class CMakeBuild(build_ext):
versions = get_versions()
-if sys.version_info < (3,0):
- raise RuntimeError("Python 3.3 or larger required.")
+if sys.version_info < (3,4):
+ raise RuntimeError("Python 3.4 or larger required.")
with open('README.rst', 'r') as descfile:
long_description = descfile.read()
@@ -128,7 +131,7 @@ setup(
maintainer='Sarah Hoffmann',
maintainer_email='lonvia at denofr.de',
download_url='https://github.com/osmcode/pyosmium',
- url='http://osmcode.org/pyosmium',
+ url='https://osmcode.org/pyosmium',
keywords=["OSM", "OpenStreetMap", "Osmium"],
license='BSD',
scripts=['tools/pyosmium-get-changes', 'tools/pyosmium-up-to-date'],
@@ -141,6 +144,7 @@ setup(
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
+ "Programming Language :: Python :: 3.9",
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: C++",
],
=====================================
src/osmium/helper.py
=====================================
@@ -19,4 +19,3 @@ def make_simple_handler(node=None, way=None, relation=None, area=None):
__HandlerWithCallbacks.area = staticmethod(area)
return __HandlerWithCallbacks()
-
=====================================
src/osmium/osm/__init__.py
=====================================
@@ -1,5 +1,5 @@
-from ._osm import *
import osmium.osm.mutable
+from ._osm import *
def create_mutable_node(node, **args):
""" Create a mutable node replacing the properties given in the
@@ -26,59 +26,72 @@ Node.replace = create_mutable_node
Way.replace = create_mutable_way
Relation.replace = create_mutable_relation
-Location.__repr__ = lambda l : 'osmium.osm.Location(x=%r, y=%r)' \
- % (l.x, l.y) \
- if l.valid() else 'osmium.osm.Location()'
-Location.__str__ = lambda l : '%f/%f' % (l.lon_without_check(), l.lat_without_check()) \
- if l.valid() else 'invalid'
+def _make_repr(attr_list):
+ fmt_string = 'osmium.osm.{0}('\
+ + ', '.join(['{0}={{1.{0}!r}}'.format(x) for x in attr_list])\
+ + ')'
-Box.__repr__ = lambda b : 'osmium.osm.Box(bottom_left=%r, top_right=%r)' \
- % (b.bottom_left, b.top_right)
-Box.__str__ = lambda b : '(%s %s)' % (b.bottom_left, b.top_right)
+ return lambda o: fmt_string.format(o.__class__.__name__, o)
-Tag.__repr__ = lambda t : 'osmium.osm.Tag(k=%r, v=%r)' % (t.k, t.v)
+def _list_repr(obj):
+ return 'osmium.osm.{}([{}])'.format(obj.__class__.__name__,
+ ', '.join(map(repr, obj)))
+
+def _list_elipse(obj):
+ objects = ','.join(map(str, obj))
+ if len(objects) > 50:
+ objects = objects[:47] + '...'
+ return objects
+
+Location.__repr__ = lambda l: 'osmium.osm.Location(x={0.x!r}, y={0.y!r})'.format(l) \
+ if l.valid() else 'osmium.osm.Location()'
+Location.__str__ = lambda l: '{:f}/{:f}'.format(l.lon_without_check(),
+ l.lat_without_check()) \
+ if l.valid() else 'invalid'
-Tag.__str__ = lambda t : '%s=%s' % (t.k, t.v)
+Box.__repr__ = _make_repr(['bottom_left', 'top_right'])
+Box.__str__ = lambda b: '({0.bottom_left!s} {0.top_right!s})'.format(b)
-TagList.__repr__ = lambda t : "osmium.osm.TagList({%s})" \
- % ",".join(["%r=%r" % (i.k, i.v) for i in t])
-TagList.__str__ = lambda t : "{%s}" % ",".join([str(i) for i in t])
+Tag.__repr__ = _make_repr(['k', 'v'])
+Tag.__str__ = lambda t: '{0.k}={0.v}'.format(t)
-NodeRef.__repr__ = lambda n : 'osmium.osm.NodeRef(ref=%r, location=%r)' % (n.ref, n.location)
-NodeRef.__str__ = lambda n : '%s@%s' % (n.ref, n.location) if n.location.valid() \
- else str(n.ref)
+TagList.__repr__ = lambda t: "osmium.osm.TagList({%s})" \
+ % ', '.join(["%r: %r" % (i.k, i.v) for i in t])
+TagList.__str__ = lambda t: '{' + _list_elipse(t) + '}'
-NodeRefList.__repr__ = lambda t : "%s([%s])" % (t.__class__.__name__,
- ",".join([repr(i) for i in t]))
-NodeRefList.__str__ = lambda t : "[%s]" % ",".join([str(i) for i in t])
+NodeRef.__repr__ = _make_repr(['ref', 'location'])
+NodeRef.__str__ = lambda n: '{0.ref:d}@{0.location!s}'.format(n) \
+ if n.location.valid() else str(n.ref)
-RelationMember.__repr__ = lambda r : 'osmium.osm.RelationMember(ref=%r, type=%r, role=%r)' \
- % (r.ref, r.type, r.role)
-RelationMember.__str__ = lambda r : '%s%d@%s' % (r.type, r.ref, r.role) \
- if r.role else '%s%d' % (r.type, r.ref)
+NodeRefList.__repr__ = _list_repr
+NodeRefList.__str__ = lambda o: '[' + _list_elipse(o) + ']'
-RelationMemberList.__repr__ = lambda t : "osmium.osm.RelationMemberList([%s])" \
- % ",".join([repr(i) for i in t])
-RelationMemberList.__str__ = lambda t : "[%s]" % ",".join([str(i) for i in t])
+RelationMember.__repr__ = _make_repr(['ref', 'type', 'role'])
+RelationMember.__str__ = lambda r: ('{0.type}{0.ref:d}@{0.role}' \
+ if r.role else '{0.type}{0.ref:d}').format(r)
-OSMObject.__repr__ = lambda o : '%s(id=%r, deleted=%r, visible=%r, version=%r, changeset=%r, uid=%r, timestamp=%r, user=%r, tags=%r)'% (o.__class__.__name__, o.id, o.deleted, o.visible, o.version, o.changeset, o.uid, o.timestamp, o.user, o.tags)
+RelationMemberList.__repr__ = _list_repr
+RelationMemberList.__str__ = lambda o: '[' + _list_elipse(o) + ']'
-def _str_ellipse(s, length=50):
- s = str(s)
- return s if len(s) <= length else (s[:length - 4] + '...' + s[-1])
+OSMObject.__repr__ = _make_repr(['id', 'deleted', 'visible', 'version', 'changeset',
+ 'uid', 'timestamp', 'user', 'tags'])
-Node.__repr__ = lambda o : '%s(id=%r, deleted=%r, visible=%r, version=%r, changeset=%r, uid=%r, timestamp=%r, user=%r, tags=%r, location=%r)'% (o.__class__.__name__, o.id, o.deleted, o.visible, o.version, o.changeset, o.uid, o.timestamp, o.user, o.tags, o.location)
-Node.__str__ = lambda n : 'n%d: location=%s tags=%s' \
- % (n.id, n.location, _str_ellipse(n.tags))
+Node.__repr__ = _make_repr(['id', 'deleted', 'visible', 'version', 'changeset',
+ 'uid', 'timestamp', 'user', 'tags', 'location'])
+Node.__str__ = lambda n: 'n{0.id:d}: location={0.location!s} tags={0.tags!s}'\
+ .format(n)
-Way.__repr__ = lambda o : '%s(id=%r, deleted=%r, visible=%r, version=%r, changeset=%r, uid=%r, timestamp=%r, user=%r, tags=%r, nodes=%r)'% (o.__class__.__name__, o.id, o.deleted, o.visible, o.version, o.changeset, o.uid, o.timestamp, o.user, o.tags, o.nodes)
-Way.__str__ = lambda o : 'w%d: nodes=%s tags=%s' \
- % (o.id, _str_ellipse(o.nodes), _str_ellipse(o.tags))
+Way.__repr__ = _make_repr(['id', 'deleted', 'visible', 'version', 'changeset',
+ 'uid', 'timestamp', 'user', 'tags', 'nodes'])
+Way.__str__ = lambda o: 'w{0.id:d}: nodes={0.nodes!s} tags={0.tags!s}' \
+ .format(o)
-Relation.__repr__ = lambda o : '%s(id=%r, deleted=%r, visible=%r, version=%r, changeset=%r, uid=%r, timestamp=%r, user=%r, tags=%r, members=%r)'% (o.__class__.__name__, o.id, o.deleted, o.visible, o.version, o.changeset, o.uid, o.timestamp, o.user, o.tags, o.members)
-Relation.__str__ = lambda o : 'r%d: members=%s, tags=%s' \
- % (o.id, _str_ellipse(o.members), _str_ellipse(o.tags))
+Relation.__repr__ = _make_repr(['id', 'deleted', 'visible', 'version', 'changeset',
+ 'uid', 'timestamp', 'user', 'tags', 'members'])
+Relation.__str__ = lambda o: 'r{0.id:d}: members={0.members!s}, tags={0.tags!s}' \
+ .format(o)
-Changeset.__repr__ = lambda o : '%s(id=%r, uid=%r, created_at=%r, closed_at=%r, open=%r, num_changes=%r, bounds=%r, user=%r, tags=%s)' %(o.__class__.__name__, o.id, o.uid, o.created_at, o.closed_at, o.open, o.num_changes, o.bounds, o.user, o.tags)
-Changeset.__str__ = lambda o : 'c%d: closed_at=%s, bounds=%s, tags=%s' \
- % (o.id, o.closed_at, o.bounds, _str_ellipse(o.tags))
+Changeset.__repr__ = _make_repr(['id', 'uid', 'created_at', 'closed_at', 'open',
+ 'num_changes', 'bounds', 'user', 'tags'])
+Changeset.__str__ = lambda o: 'c{0.id:d}: closed_at={0.closed_at!s}, bounds={0.bounds!s}, tags={0.tags!s}' \
+ .format(o)
=====================================
src/osmium/osm/mutable.py
=====================================
@@ -10,7 +10,7 @@ class OSMObject(object):
"""
def __init__(self, base=None, id=None, version=None, visible=None, changeset=None,
- timestamp=None, uid=None, tags=None, user=None):
+ timestamp=None, uid=None, tags=None, user=None):
if base is None:
self.id = id
self.version = version
@@ -73,5 +73,3 @@ class Relation(OSMObject):
self.members = members
else:
self.members = members if members is not None else base.members
-
-
=====================================
src/osmium/replication/server.py
=====================================
@@ -1,14 +1,7 @@
""" Helper functions to communicate with replication servers.
"""
-try:
- import urllib.request as urlrequest
-except ImportError:
- import urllib2 as urlrequest
-try:
- import urllib.error as urlerror
-except ImportError:
- import urllib2 as urlerror
+import urllib.request as urlrequest
import datetime as dt
from collections import namedtuple
from math import ceil
@@ -18,13 +11,13 @@ from osmium import version
import logging
-log = logging.getLogger('pyosmium')
-log.addHandler(logging.NullHandler())
+LOG = logging.getLogger('pyosmium')
+LOG.addHandler(logging.NullHandler())
OsmosisState = namedtuple('OsmosisState', ['sequence', 'timestamp'])
DownloadResult = namedtuple('DownloadResult', ['id', 'reader', 'newest'])
-class ReplicationServer(object):
+class ReplicationServer:
""" Represents a server that publishes replication data. Replication
change files allow to keep local OSM data up-to-date without downloading
the full dataset again.
@@ -88,7 +81,7 @@ class ReplicationServer(object):
try:
diffdata = self.get_diff_block(current_id)
except:
- log.debug("Error during diff download. Bailing out.")
+ LOG.debug("Error during diff download. Bailing out.")
diffdata = ''
if len(diffdata) == 0:
if start_id == current_id:
@@ -96,8 +89,8 @@ class ReplicationServer(object):
break
left_size -= rd.add_buffer(diffdata, self.diff_type)
- log.debug("Downloaded change %d. (%d kB available in download buffer)"
- % (current_id, left_size / 1024))
+ LOG.debug("Downloaded change %d. (%d kB available in download buffer)",
+ current_id, left_size / 1024)
current_id += 1
return DownloadResult(current_id - 1, rd, newest.sequence)
@@ -137,7 +130,7 @@ class ReplicationServer(object):
return diffs.id
def apply_diffs_to_file(self, infile, outfile, start_id, max_size=1024,
- set_replication_header=True, extra_headers={},
+ set_replication_header=True, extra_headers=None,
outformat=None):
""" Download diffs starting with sequence id `start_id`, merge them
with the data from the OSM file named `infile` and write the result
@@ -178,17 +171,19 @@ class ReplicationServer(object):
h.set("osmosis_replication_sequence_number", str(diffs.id))
info = self.get_state_info(diffs.id)
h.set("osmosis_replication_timestamp", info.timestamp.strftime("%Y-%m-%dT%H:%M:%SZ"))
- for k,v in extra_headers.items():
- h.set(k, v)
+ if extra_headers is not None:
+ for k, v in extra_headers.items():
+ h.set(k, v)
if outformat is None:
of = oio.File(outfile)
else:
of = oio.File(outfile, outformat)
+ of.has_multiple_object_versions = has_history
writer = oio.Writer(of, h)
- log.debug("Merging changes into OSM file.")
+ LOG.debug("Merging changes into OSM file.")
diffs.reader.apply_to_reader(reader, writer, has_history)
@@ -220,7 +215,7 @@ class ReplicationServer(object):
lower = None
lowerid = 0
while lower is None:
- log.debug("Trying with Id %s" % lowerid)
+ LOG.debug("Trying with Id %s", lowerid)
lower = self.get_state_info(lowerid)
if lower is not None and lower.timestamp >= timestamp:
@@ -287,11 +282,11 @@ class ReplicationServer(object):
returns `None`. `retries` sets the number of times the download
is retried when pyosmium detects a truncated state file.
"""
- for retry in range(retries + 1):
+ for _ in range(retries + 1):
try:
response = self.open_url(self.make_request(self.get_state_url(seq)))
except Exception as err:
- log.debug("Loading state info {} failed with: {}".format(seq, str(err)))
+ LOG.debug("Loading state info %s failed with: %s", seq, str(err))
return None
ts = None
@@ -341,13 +336,14 @@ class ReplicationServer(object):
if seq is None:
return self.baseurl + '/state.txt'
- return '%s/%03i/%03i/%03i.state.txt' % (self.baseurl,
- seq / 1000000, (seq % 1000000) / 1000, seq % 1000)
+ return '%s/%03i/%03i/%03i.state.txt' % \
+ (self.baseurl, seq / 1000000, (seq % 1000000) / 1000, seq % 1000)
def get_diff_url(self, seq):
""" Returns the URL to the diff file for the given sequence id.
"""
- return '%s/%03i/%03i/%03i.%s' % (self.baseurl,
- seq / 1000000, (seq % 1000000) / 1000, seq % 1000,
- self.diff_type)
+ return '%s/%03i/%03i/%03i.%s' % \
+ (self.baseurl,
+ seq / 1000000, (seq % 1000000) / 1000, seq % 1000,
+ self.diff_type)
=====================================
src/osmium/replication/utils.py
=====================================
@@ -6,10 +6,10 @@ from collections import namedtuple
from osmium.io import Reader as oreader
from osmium.osm import NOTHING
-log = logging.getLogger('pyosmium')
+LOG = logging.getLogger('pyosmium')
ReplicationHeader = namedtuple('ReplicationHeader',
- ['url', 'sequence', 'timestamp'])
+ ['url', 'sequence', 'timestamp'])
def get_replication_header(fname):
""" Scans the given file for an Osmosis replication header. It returns
@@ -28,21 +28,21 @@ def get_replication_header(fname):
url = h.get("osmosis_replication_base_url")
if url or ts:
- log.debug("Replication information found in OSM file header.")
+ LOG.debug("Replication information found in OSM file header.")
if url:
- log.debug("Replication URL: %s" % url)
+ LOG.debug("Replication URL: %s", url)
# the sequence ID is only considered valid, if an URL is given
seq = h.get("osmosis_replication_sequence_number")
if seq:
- log.debug("Replication sequence: %s" % seq)
+ LOG.debug("Replication sequence: %s", seq)
try:
seq = int(seq)
if seq < 0:
- log.warning("Sequence id '%d' in OSM file header is negative. Ignored." % seq)
+ LOG.warning("Sequence id '%d' in OSM file header is negative. Ignored.", seq)
seq = None
except ValueError:
- log.warning("Sequence id '%s' in OSM file header is not a number.Ignored" % seq)
+ LOG.warning("Sequence id '%s' in OSM file header is not a number. Ignored.", seq)
seq = None
else:
seq = None
@@ -51,13 +51,13 @@ def get_replication_header(fname):
seq = None
if ts:
- log.debug("Replication timestamp: %s" % ts)
+ LOG.debug("Replication timestamp: %s", ts)
try:
ts = dt.datetime.strptime(ts, "%Y-%m-%dT%H:%M:%SZ")
ts = ts.replace(tzinfo=dt.timezone.utc)
except ValueError:
- log.warning("Date in OSM file header is not in ISO8601 format (e.g. 2015-12-24T08:08Z). Ignored")
+ LOG.warning("Date in OSM file header is not in ISO8601 format (e.g. 2015-12-24T08:08Z). Ignored.")
ts = None
else:
ts = None
=====================================
src/osmium/version.py
=====================================
@@ -3,13 +3,13 @@ Version information.
"""
# the major version
-pyosmium_major = '3.0'
+pyosmium_major = '3.1'
# current release (Pip version)
-pyosmium_release = '3.0.1'
+pyosmium_release = '3.1.2'
# libosmium version shipped with the Pip release
-libosmium_version = '2.15.6'
+libosmium_version = '2.16.0'
# protozero version shipped with the Pip release
protozero_version = '1.7.0'
# pybind11 version shipped with the Pip release
-pybind11_version = '2.5.0'
+pybind11_version = '2.6.1'
=====================================
test/test_replication.py
=====================================
@@ -31,15 +31,15 @@ class UrllibMock(MagicMock):
self.side_effect = [BytesIO(dedent(s).encode()) for s in files]
def test_get_state_url():
- svr = rserv.ReplicationServer("http://text.org")
+ svr = rserv.ReplicationServer("https://text.org")
data = [
- (None, 'http://text.org/state.txt'),
- (1, 'http://text.org/000/000/001.state.txt'),
- (999, 'http://text.org/000/000/999.state.txt'),
- (1000, 'http://text.org/000/001/000.state.txt'),
- (573923, 'http://text.org/000/573/923.state.txt'),
- (3290012, 'http://text.org/003/290/012.state.txt'),
+ (None, 'https://text.org/state.txt'),
+ (1, 'https://text.org/000/000/001.state.txt'),
+ (999, 'https://text.org/000/000/999.state.txt'),
+ (1000, 'https://text.org/000/001/000.state.txt'),
+ (573923, 'https://text.org/000/573/923.state.txt'),
+ (3290012, 'https://text.org/003/290/012.state.txt'),
]
for i, o in data:
@@ -69,7 +69,7 @@ def test_get_state_valid(mock):
txnMax=1219304113
txnActiveList=1219303583,1219304054,1219304104""")
- res = rserv.ReplicationServer("http://test.io").get_state_info()
+ res = rserv.ReplicationServer("https://test.io").get_state_info()
assert_is_not_none(res)
assert_equals(res.timestamp, mkdate(2017, 8, 26, 11, 4, 2))
@@ -89,7 +89,7 @@ def test_get_state_sequence_cut(mock):
sequenceNumber=2594669
timestamp=2017-08-26T11\:04\:02Z"""))
- res = rserv.ReplicationServer("http://test.io").get_state_info()
+ res = rserv.ReplicationServer("https://test.io").get_state_info()
assert_is_not_none(res)
assert_equals(res.timestamp, mkdate(2017, 8, 26, 11, 4, 2))
@@ -110,7 +110,7 @@ def test_get_state_date_cut(mock):
sequenceNumber=2594669
timestamp=2017-08-26T11\:04\:02Z"""))
- res = rserv.ReplicationServer("http://test.io").get_state_info()
+ res = rserv.ReplicationServer("https://test.io").get_state_info()
assert_is_not_none(res)
assert_equals(res.timestamp, mkdate(2017, 8, 26, 11, 4, 2))
@@ -131,7 +131,7 @@ def test_get_state_timestamp_cut(mock):
sequenceNumber=2594669
timestamp=2017-08-26T11\:04\:02Z"""))
- res = rserv.ReplicationServer("http://test.io").get_state_info()
+ res = rserv.ReplicationServer("https://test.io").get_state_info()
assert_is_not_none(res)
assert_equals(res.timestamp, mkdate(2017, 8, 26, 11, 4, 2))
@@ -162,7 +162,7 @@ def test_get_state_too_many_retries(mock):
sequenceNumber=2594669
timestamp=2017-08-26T11\:04\:02Z"""))
- res = rserv.ReplicationServer("http://test.io").get_state_info()
+ res = rserv.ReplicationServer("https://test.io").get_state_info()
assert_is_none(res)
@@ -174,7 +174,7 @@ def test_get_state_too_many_retries(mock):
def test_get_state_server_timeout(mock):
mock.side_effect = URLError(reason='Mock')
- svr = rserv.ReplicationServer("http://test.io")
+ svr = rserv.ReplicationServer("https://test.io")
assert_is_none(svr.get_state_info())
@patch('osmium.replication.server.urlrequest.urlopen', new_callable=UrllibMock)
@@ -187,7 +187,7 @@ def test_apply_diffs_count(mock):
w1
r1
"""))
- svr = rserv.ReplicationServer("http://test.io", "opl")
+ svr = rserv.ReplicationServer("https://test.io", "opl")
h = CountingHandler()
assert_equals(100, svr.apply_diffs(h, 100, 10000))
@@ -205,7 +205,7 @@ def test_apply_diffs_without_simplify(mock):
w1
r1
"""))
- svr = rserv.ReplicationServer("http://test.io", "opl")
+ svr = rserv.ReplicationServer("https://test.io", "opl")
h = CountingHandler()
assert_equals(100, svr.apply_diffs(h, 100, 10000, simplify=False))
@@ -222,7 +222,7 @@ def test_apply_diffs_with_simplify(mock):
w1
r1
"""))
- svr = rserv.ReplicationServer("http://test.io", "opl")
+ svr = rserv.ReplicationServer("https://test.io", "opl")
h = CountingHandler()
assert_equals(100, svr.apply_diffs(h, 100, 10000, simplify=True))
@@ -237,7 +237,7 @@ def test_apply_with_location(mock):
n1 x10.0 y23.0
w1 Nn1,n2
"""))
- svr = rserv.ReplicationServer("http://test.io", "opl")
+ svr = rserv.ReplicationServer("https://test.io", "opl")
class Handler(CountingHandler):
def way(self, w):
@@ -267,7 +267,7 @@ def test_apply_reader_without_simplify(mock):
w1
r1
"""))
- svr = rserv.ReplicationServer("http://test.io", "opl")
+ svr = rserv.ReplicationServer("https://test.io", "opl")
h = CountingHandler()
@@ -288,7 +288,7 @@ def test_apply_reader_with_simplify(mock):
w1
r1
"""))
- svr = rserv.ReplicationServer("http://test.io", "opl")
+ svr = rserv.ReplicationServer("https://test.io", "opl")
h = CountingHandler()
diffs = svr.collect_diffs(100, 100000)
@@ -306,7 +306,7 @@ def test_apply_reader_with_location(mock):
n1 x10.0 y23.0
w1 Nn1,n2
"""))
- svr = rserv.ReplicationServer("http://test.io", "opl")
+ svr = rserv.ReplicationServer("https://test.io", "opl")
class Handler(CountingHandler):
def way(self, w):
=====================================
tools/pyosmium-get-changes
=====================================
@@ -27,11 +27,11 @@ cookies to the server and will save received cookies to the jar file.
from argparse import ArgumentParser, RawDescriptionHelpFormatter, ArgumentTypeError
import datetime as dt
-from sys import version_info as python_version
import socket
from osmium.replication import server as rserv
from osmium.replication import newest_change_from_file
from osmium.replication.utils import get_replication_header
+from osmium.version import pyosmium_release
from osmium import SimpleHandler, WriteHandler
try:
@@ -85,8 +85,7 @@ class ReplicationStart(object):
def from_date(datestr):
try:
date = dt.datetime.strptime(datestr, "%Y-%m-%dT%H:%M:%SZ")
- if python_version >= (3,0):
- date = date.replace(tzinfo=dt.timezone.utc)
+ date = date.replace(tzinfo=dt.timezone.utc)
except ValueError:
raise ArgumentTypeError("Date needs to be in ISO8601 format (e.g. 2015-12-24T08:08:08Z).")
@@ -133,7 +132,7 @@ def get_arg_parser(from_main=False):
usage=None if from_main else 'pyosmium-get-changes [options]',
formatter_class=RawDescriptionHelpFormatter)
parser.add_argument('-v', dest='loglevel', action='count', default=0,
- help='Increase verbosity')
+ help='Increase verbosity (can be used multiple times)')
parser.add_argument('-o', '--outfile', dest='outfile',
help=h("""Name of diff output file. If omitted, only the
sequence ID will be printed where updates would start."""))
@@ -166,16 +165,18 @@ def get_arg_parser(from_main=False):
ignore potential replication information in the
header and search for the newest OSM object."""))
parser.add_argument('-d', '--no-deduplicate', action='store_false', dest='simplify',
- help='Do not deduplicate and sort diffs.')
+ help='Do not deduplicate diffs.')
parser.add_argument('--socket-timeout', dest='socket_timeout', type=int, default=60,
help='Set timeout for file downloads.')
+ parser.add_argument('--version', action='version', version='pyosmium ' + pyosmium_release)
return parser
def main(args):
logging.basicConfig(stream=sys.stderr,
- format='%(levelname)s: %(message)s')
+ format='%(asctime)s %(levelname)s: %(message)s',
+ datefmt='%Y-%m-%d %H:%M:%S')
options = get_arg_parser(from_main=True).parse_args(args)
=====================================
tools/pyosmium-up-to-date
=====================================
@@ -15,6 +15,9 @@ information from the osmosis headers. That means that after the first update,
subsequent calls to pyosmium-up-to-date will continue the updates from the same
server exactly where they have left of.
+This program can update normal OSM data files as well as OSM history files.
+It detects automatically on what type of file it is called.
+
The program returns 0, if updates have been successfully applied up to
the newest data. It returns 1, if some updates have been applied but
there is still data available on the server (either because the size
@@ -35,7 +38,6 @@ import socket
from argparse import ArgumentParser, RawDescriptionHelpFormatter
import datetime as dt
-from sys import version_info as python_version
from osmium.replication import server as rserv
from osmium.replication.utils import get_replication_header
from osmium.replication import newest_change_from_file
@@ -94,8 +96,7 @@ def update_from_custom_server(url, seq, ts, options):
if not options.force_update:
cmpdate = dt.datetime.utcnow() - dt.timedelta(days=90)
- if python_version >= (3,0):
- cmpdate = cmpdate.replace(tzinfo=dt.timezone.utc)
+ cmpdate = cmpdate.replace(tzinfo=dt.timezone.utc)
if ts < cmpdate:
log.error(
"""The OSM file is more than 3 months old. You should download a
@@ -116,18 +117,25 @@ def update_from_custom_server(url, seq, ts, options):
else:
ofname = outfile
- extra_headers = { 'generator' : 'pyosmium-up-to-date/' + pyosmium_release }
- outseqs = svr.apply_diffs_to_file(infile, ofname, startseq,
- max_size=options.outsize*1024,
- extra_headers=extra_headers,
- outformat=options.outformat)
-
- if outseqs is None:
- log.info("No new updates found.")
- return 3
+ try:
+ extra_headers = { 'generator' : 'pyosmium-up-to-date/' + pyosmium_release }
+ outseqs = svr.apply_diffs_to_file(infile, ofname, startseq,
+ max_size=options.outsize*1024,
+ extra_headers=extra_headers,
+ outformat=options.outformat)
+
+ if outseqs is None:
+ log.info("No new updates found.")
+ return 3
- if outfile is None:
- os.rename(ofname, infile)
+ if outfile is None:
+ os.rename(ofname, infile)
+ finally:
+ if outfile is None:
+ try:
+ os.remove(ofname)
+ except FileNotFoundError:
+ pass
log.info("Downloaded until %d. Server has data available until %d." % outseqs)
@@ -177,7 +185,7 @@ def get_arg_parser(from_main=False):
usage=None if from_main else 'pyosmium-up-to-date [options] <osm file>',
formatter_class=RawDescriptionHelpFormatter)
parser.add_argument('-v', dest='loglevel', action='count', default=0,
- help='Increase verbosity')
+ help='Increase verbosity (can be used multiple times)')
parser.add_argument('infile', metavar='<osm file>', help="OSM file to update")
parser.add_argument('-o', '--outfile', dest='outfile',
help=h("""Name output of file. If missing, the input file
@@ -209,6 +217,7 @@ def get_arg_parser(from_main=False):
received cookies will be written to.""")
parser.add_argument('--socket-timeout', dest='socket_timeout', type=int, default=60,
help='Set timeout for file downloads.')
+ parser.add_argument('--version', action='version', version='pyosmium ' + pyosmium_release)
return parser
@@ -217,7 +226,8 @@ def open_with_cookie(url):
if __name__ == '__main__':
logging.basicConfig(stream=sys.stderr,
- format='%(asctime)s %(levelname)s: %(message)s')
+ format='%(asctime)s %(levelname)s: %(message)s',
+ datefmt='%Y-%m-%d %H:%M:%S')
options = get_arg_parser(from_main=True).parse_args()
log.setLevel(max(3 - options.loglevel, 0) * 10)
View it on GitLab: https://salsa.debian.org/debian-gis-team/pyosmium/-/compare/d3ae7138780855bc55bdc8dee8178d2d7e64f06f...8a74dca23a58efe0b4b3ea7916f544bbdd8f2c4f
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/pyosmium/-/compare/d3ae7138780855bc55bdc8dee8178d2d7e64f06f...8a74dca23a58efe0b4b3ea7916f544bbdd8f2c4f
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20210119/bb4e814d/attachment-0001.html>
More information about the Pkg-grass-devel
mailing list