[Git][debian-gis-team/tilemaker][patch-queue/master] 9 commits: New upstream version 2.1.0
ǝɹʇʇɐʃǝ◖ xıʃǝɟ (@pantierra)
gitlab at salsa.debian.org
Thu Mar 9 14:42:54 GMT 2023
ǝɹʇʇɐʃǝ◖ xıʃǝɟ pushed to branch patch-queue/master at Debian GIS Project / tilemaker
Commits:
9a0c5e63 by Felix Delattre at 2022-08-02T18:41:10+00:00
New upstream version 2.1.0
- - - - -
685fc3b9 by Felix Delattre at 2022-08-02T19:25:20+00:00
New upstream version 2.2.0
- - - - -
92635990 by Felix Delattre at 2022-08-09T15:23:41+00:00
Added patch to include latomic flag when needed.
- - - - -
98a8e269 by Felix Delattre at 2022-08-09T16:08:56+00:00
Set distribution back to unstable.
- - - - -
aaebeae7 by Bas Couwenberg at 2022-11-29T18:41:52+01:00
Add Rules-Requires-Root to control file.
- - - - -
5bc10427 by Bas Couwenberg at 2023-01-18T17:21:00+01:00
Bump Standards-Version to 4.6.2, no changes.
- - - - -
f4e10bf7 by Felix Delattre at 2023-03-09T09:50:30+00:00
New upstream version 2.3.0
- - - - -
65a27639 by Felix Delattre at 2023-03-09T09:52:26+00:00
Update upstream source from tag 'upstream/2.3.0'
Update to upstream version '2.3.0'
with Debian dir 940c1e1b2ded91a4e89e9810bbcf08664f25f678
- - - - -
3310c439 by Felix Delattre at 2023-03-09T09:53:00+00:00
New upstream release.
- - - - -
30 changed files:
- .github/workflows/ci.yml
- CHANGELOG.md
- CMakeLists.txt
- Makefile
- README.md
- + cmake/CheckCxxAtomic.cmake
- debian/changelog
- debian/control
- debian/patches/0001-Fix-manpage-in-makefiles.patch
- + debian/patches/0002-Check-and-add-latomic.patch
- debian/patches/series
- debian/rules
- docs/CONFIGURATION.md
- include/coordinates.h
- include/geom.h
- include/osm_lua_processing.h
- include/osm_store.h
- include/output_object.h
- include/shp_mem_tiles.h
- server/server.rb
- src/coordinates.cpp
- src/osm_lua_processing.cpp
- src/osm_store.cpp
- src/output_object.cpp
- src/read_pbf.cpp
- src/shared_data.cpp
- src/shp_mem_tiles.cpp
- src/tile_worker.cpp
- src/tilemaker.cpp
- src/write_geometry.cpp
Changes:
=====================================
.github/workflows/ci.yml
=====================================
@@ -52,7 +52,7 @@ jobs:
strategy:
matrix:
include:
- - os: ubuntu-18.04
+ - os: ubuntu-22.04
triplet: x64-linux
executable: tilemaker
path: /usr/local/share/vcpkg/installed
=====================================
CHANGELOG.md
=====================================
@@ -1,5 +1,29 @@
# Changelog
+# [2.3.0] - 2023-03-08
+
+### Added
+- Send project name to init_function (@systemed)
+- Remove zero-width spikes after simplification (@systemed)
+- Remove multipolygon inners below filter area size (@systemed)
+
+### Changed
+- Move centroid and "no indexed layer" errors to verbose mode only (@systemed)
+- Report missing layers consistently (@akx)
+- Update Ruby server to Rack 3 (@typebrook)
+- Move mmap shutdown to end of PBF reading (@systemed)
+
+### Fixed
+- Use std::ofstream instead of boost::filesystem (@milovanderlinden)
+- Scale geometries before simplifying to avoid reintroducing self-intersections (@systemed)
+- Fix manpage in makefiles (@xamanu)
+- Intersect multipolygons part-by-part with clipping box to fix Boost.Geometry issue (@systemed)
+- Windows issues (@roundby)
+- Add libatomic for rare architectures (@xamanu)
+- Ignore nodes in ways with --skip-integrity (@systemed)
+- Correctly mask IDs for output with include_ids (@systemed)
+
+
## [2.2.0] - 2022-03-11
### Added
=====================================
CMakeLists.txt
=====================================
@@ -110,8 +110,16 @@ add_executable(tilemaker vector_tile.pb.cc osmformat.pb.cc ${tilemaker_src_files
target_link_libraries(tilemaker ${PROTOBUF_LIBRARY} ${LIBSHP_LIBRARIES} ${SQLITE3_LIBRARIES} ${LUAJIT_LIBRARY} ${LUA_LIBRARIES} ${ZLIB_LIBRARY} ${THREAD_LIB} ${CMAKE_DL_LIBS}
Boost::system Boost::filesystem Boost::program_options Boost::iostreams)
+include(CheckCxxAtomic)
+if(NOT HAVE_CXX11_ATOMIC)
+ string(APPEND CMAKE_CXX_STANDARD_LIBRARIES
+ " ${LIBATOMIC_LINK_FLAGS}")
+endif()
+
if(MSVC)
target_link_libraries(tilemaker unofficial::sqlite3::sqlite3)
endif()
+install(FILES docs/man/tilemaker.1 DESTINATION share/man/man1)
+
install(TARGETS tilemaker RUNTIME DESTINATION bin)
=====================================
Makefile
=====================================
@@ -101,8 +101,8 @@ tilemaker: include/osmformat.pb.o include/vector_tile.pb.o src/mbtiles.o src/pbf
install:
install -m 0755 -d $(DESTDIR)$(prefix)/bin/
install -m 0755 tilemaker $(DESTDIR)$(prefix)/bin/
- install -d ${DESTDIR}${MANPREFIX}/man1/
- -install docs/man/tilemaker.1 ${DESTDIR}${MANPREFIX}/man1/
+ install -m 0755 -d ${DESTDIR}${MANPREFIX}/man1/
+ install docs/man/tilemaker.1 ${DESTDIR}${MANPREFIX}/man1/
clean:
rm -f tilemaker src/*.o include/*.o include/*.pb.h
=====================================
README.md
=====================================
@@ -44,7 +44,7 @@ Then, to serve your tiles using the demonstration server:
You can now navigate to http://localhost:8080/ and see your map!
-(If you don't already have them, you'll need to install Ruby and the required gems to run the demonstration server. On Ubuntu, for example, `sudo apt install sqlite3 libsqlite3-dev ruby ruby-dev` and then `sudo gem install sqlite3 cgi glug rack`.)
+(If you don't already have them, you'll need to install Ruby and the required gems to run the demonstration server. On Ubuntu, for example, `sudo apt install sqlite3 libsqlite3-dev ruby ruby-dev` and then `sudo gem install sqlite3 cgi glug rack rackup`.)
## Your own configuration
=====================================
cmake/CheckCxxAtomic.cmake
=====================================
@@ -0,0 +1,66 @@
+# some platforms do not offer support for atomic primitive for all integer
+# types, in that case we need to link against libatomic
+
+include(CheckCXXSourceCompiles)
+include(CMakePushCheckState)
+
+
+function(check_cxx_atomics var)
+ set(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} -std=c++11")
+ check_cxx_source_compiles("
+#include <atomic>
+#include <cstdint>
+#include <cstddef>
+#if defined(__SIZEOF_INT128__)
+// Boost needs 16-byte atomics for tagged pointers.
+// These are implemented via inline instructions on the platform
+// if 16-byte alignment can be proven, and are delegated to libatomic
+// library routines otherwise. Whether or not alignment is provably
+// OK for a std::atomic unfortunately depends on compiler version and
+// optimization levels, and also on the details of the expression.
+// We specifically test access via an otherwise unknown pointer here
+// to ensure we get the most complex case. If this access can be
+// done without libatomic, then all accesses can be done.
+struct tagged_ptr {
+ int* ptr;
+ std::size_t tag;
+};
+void atomic16(std::atomic<tagged_ptr> *ptr)
+{
+ tagged_ptr p{nullptr, 1};
+ ptr->store(p);
+ tagged_ptr f = ptr->load();
+ tagged_ptr new_tag{nullptr, 0};
+ ptr->compare_exchange_strong(f, new_tag);
+}
+#endif
+int main() {
+#if defined(__SIZEOF_INT128__)
+ std::atomic<tagged_ptr> ptr;
+ atomic16(&ptr);
+#endif
+ std::atomic<uint8_t> w1;
+ std::atomic<uint16_t> w2;
+ std::atomic<uint32_t> w4;
+ std::atomic<uint64_t> w8;
+ return w1 + w2 + w4 + w8;
+}
+" ${var})
+endfunction(check_cxx_atomics)
+
+cmake_push_check_state()
+check_cxx_atomics(HAVE_CXX11_ATOMIC)
+cmake_pop_check_state()
+
+if(NOT HAVE_CXX11_ATOMIC)
+ cmake_push_check_state()
+ set(CMAKE_REQUIRED_LIBRARIES "atomic")
+ check_cxx_atomics(HAVE_LIBATOMIC)
+ cmake_pop_check_state()
+ if(HAVE_LIBATOMIC)
+ set(LIBATOMIC_LINK_FLAGS "-latomic")
+ else()
+ message(FATAL_ERROR
+ "Host compiler ${CMAKE_CXX_COMPILER} requires libatomic, but it is not found")
+ endif()
+endif()
=====================================
debian/changelog
=====================================
@@ -1,8 +1,20 @@
-tilemaker (2.2.0-2) UNRELEASED; urgency=medium
+tilemaker (2.3.0-1) UNRELEASED; urgency=medium
- * Added libatomic for armel, mipsel and powerpc architectures.
+ [ Bas Couwenberg ]
+ * Team upload.
+ * Add Rules-Requires-Root to control file.
+ * Bump Standards-Version to 4.6.2, no changes.
+
+ [ Felix Delattre ]
+ * New upstream release.
+
+ -- Felix Delattre <debian at xama.nu> Thu, 09 Mar 2023 09:52:55 +0000
+
+tilemaker (2.2.0-2) unstable; urgency=medium
+
+ * Added patch to include latomic flag when needed.
- -- Felix Delattre <debian at xama.nu> Wed, 03 Aug 2022 12:57:08 +0000
+ -- Felix Delattre <debian at xama.nu> Tue, 09 Aug 2022 16:08:02 +0000
tilemaker (2.2.0-1) unstable; urgency=medium
=====================================
debian/control
=====================================
@@ -17,10 +17,11 @@ Build-Depends: cmake,
pkg-config,
protobuf-compiler,
rapidjson-dev
-Standards-Version: 4.6.1
+Standards-Version: 4.6.2
Vcs-Browser: https://salsa.debian.org/debian-gis-team/tilemaker
Vcs-Git: https://salsa.debian.org/debian-gis-team/tilemaker.git
Homepage: https://tilemaker.org/
+Rules-Requires-Root: no
Package: tilemaker
Section: utils
=====================================
debian/patches/0001-Fix-manpage-in-makefiles.patch
=====================================
@@ -1,9 +1,9 @@
From: Felix Delattre <felix at delattre.de>
Date: Tue, 2 Aug 2022 22:04:10 +0000
Subject: Fix manpage in makefile.
+
Origin: https://github.com/systemed/tilemaker/pull/423/commits/b810fd7102a43840fd60f2fac139ec745a1db5c4
Bug: https://github.com/systemed/tilemaker/pull/423
-
---
CMakeLists.txt | 1 +
Makefile | 4 ++--
=====================================
debian/patches/0002-Check-and-add-latomic.patch
=====================================
@@ -0,0 +1,106 @@
+From: Felix Delattre <felix at delattre.de>
+Date: Mon, 8 Aug 2022 19:18:34 +0000
+Subject: Check and add latomic
+
+Origin: https://github.com/systemed/tilemaker/pull/427/commits/d837f3ac82668d72ffd6877152824405cf6b124e
+Bug: https://github.com/systemed/tilemaker/pull/427
+---
+ CMakeLists.txt | 7 +++++
+ cmake/CheckCxxAtomic.cmake | 66 ++++++++++++++++++++++++++++++++++++++++++++++
+ 2 files changed, 73 insertions(+)
+ create mode 100644 cmake/CheckCxxAtomic.cmake
+
+diff --git a/CMakeLists.txt b/CMakeLists.txt
+index 78bfa5c..204f612 100644
+--- a/CMakeLists.txt
++++ b/CMakeLists.txt
+@@ -106,10 +106,17 @@ file(GLOB tilemaker_src_files
+ src/tilemaker.cpp
+ src/write_geometry.cpp
+ )
++
+ add_executable(tilemaker vector_tile.pb.cc osmformat.pb.cc ${tilemaker_src_files})
+ target_link_libraries(tilemaker ${PROTOBUF_LIBRARY} ${LIBSHP_LIBRARIES} ${SQLITE3_LIBRARIES} ${LUAJIT_LIBRARY} ${LUA_LIBRARIES} ${ZLIB_LIBRARY} ${THREAD_LIB} ${CMAKE_DL_LIBS}
+ Boost::system Boost::filesystem Boost::program_options Boost::iostreams)
+
++include(CheckCxxAtomic)
++if(NOT HAVE_CXX11_ATOMIC)
++ string(APPEND CMAKE_CXX_STANDARD_LIBRARIES
++ " ${LIBATOMIC_LINK_FLAGS}")
++endif()
++
+ if(MSVC)
+ target_link_libraries(tilemaker unofficial::sqlite3::sqlite3)
+ endif()
+diff --git a/cmake/CheckCxxAtomic.cmake b/cmake/CheckCxxAtomic.cmake
+new file mode 100644
+index 0000000..a4c8a77
+--- /dev/null
++++ b/cmake/CheckCxxAtomic.cmake
+@@ -0,0 +1,66 @@
++# some platforms do not offer support for atomic primitive for all integer
++# types, in that case we need to link against libatomic
++
++include(CheckCXXSourceCompiles)
++include(CMakePushCheckState)
++
++
++function(check_cxx_atomics var)
++ set(CMAKE_REQUIRED_FLAGS "${CMAKE_REQUIRED_FLAGS} -std=c++11")
++ check_cxx_source_compiles("
++#include <atomic>
++#include <cstdint>
++#include <cstddef>
++#if defined(__SIZEOF_INT128__)
++// Boost needs 16-byte atomics for tagged pointers.
++// These are implemented via inline instructions on the platform
++// if 16-byte alignment can be proven, and are delegated to libatomic
++// library routines otherwise. Whether or not alignment is provably
++// OK for a std::atomic unfortunately depends on compiler version and
++// optimization levels, and also on the details of the expression.
++// We specifically test access via an otherwise unknown pointer here
++// to ensure we get the most complex case. If this access can be
++// done without libatomic, then all accesses can be done.
++struct tagged_ptr {
++ int* ptr;
++ std::size_t tag;
++};
++void atomic16(std::atomic<tagged_ptr> *ptr)
++{
++ tagged_ptr p{nullptr, 1};
++ ptr->store(p);
++ tagged_ptr f = ptr->load();
++ tagged_ptr new_tag{nullptr, 0};
++ ptr->compare_exchange_strong(f, new_tag);
++}
++#endif
++int main() {
++#if defined(__SIZEOF_INT128__)
++ std::atomic<tagged_ptr> ptr;
++ atomic16(&ptr);
++#endif
++ std::atomic<uint8_t> w1;
++ std::atomic<uint16_t> w2;
++ std::atomic<uint32_t> w4;
++ std::atomic<uint64_t> w8;
++ return w1 + w2 + w4 + w8;
++}
++" ${var})
++endfunction(check_cxx_atomics)
++
++cmake_push_check_state()
++check_cxx_atomics(HAVE_CXX11_ATOMIC)
++cmake_pop_check_state()
++
++if(NOT HAVE_CXX11_ATOMIC)
++ cmake_push_check_state()
++ set(CMAKE_REQUIRED_LIBRARIES "atomic")
++ check_cxx_atomics(HAVE_LIBATOMIC)
++ cmake_pop_check_state()
++ if(HAVE_LIBATOMIC)
++ set(LIBATOMIC_LINK_FLAGS "-latomic")
++ else()
++ message(FATAL_ERROR
++ "Host compiler ${CMAKE_CXX_COMPILER} requires libatomic, but it is not found")
++ endif()
++endif()
=====================================
debian/patches/series
=====================================
@@ -1 +1,2 @@
0001-Fix-manpage-in-makefiles.patch
+0002-Check-and-add-latomic.patch
=====================================
debian/rules
=====================================
@@ -10,11 +10,7 @@ export DEB_BUILD_MAINT_OPTIONS = hardening=+all
include /usr/share/dpkg/pkg-info.mk
TM_VERSION := $(shell echo $(DEB_VERSION_UPSTREAM) | sed -e 's/\+.*//')
-ifneq (,$(filter $(DEB_BUILD_ARCH),armel mipsel powerpc))
- export DEB_CXXFLAGS_MAINT_APPEND=-DTM_VERSION=$(TM_VERSION) -latomic
-else
- export DEB_CXXFLAGS_MAINT_APPEND=-DTM_VERSION=$(TM_VERSION)
-endif
+export DEB_CXXFLAGS_MAINT_APPEND=-DTM_VERSION=$(TM_VERSION)
%:
dh $@ --buildsystem=cmake
=====================================
docs/CONFIGURATION.md
=====================================
@@ -54,6 +54,9 @@ A typical config file would look like this:
The order of layers will be carried forward into the vector tile.
+Layers with `write_to` set must appear after the layers they're writing into.
+An incorrect order will result in "the layer to write doesn't exist".
+
All options are compulsory unless stated otherwise. If tilemaker baulks at the JSON file, check everything's included, and run it through an online JSON validator to check for syntax errors.
By default tilemaker expects to find this file at config.json, but you can specify another filename with the `--config` command-line option.
@@ -106,9 +109,9 @@ For example:
Your Lua file needs to supply 5 things:
1. `node_keys`, a list of those OSM keys which indicate that a node should be processed
-2. `init_function` (optional), a function to initialize Lua logic
-2. `node_function`, a function to process an OSM node and add it to layers
-3. `way_function`, a function to process an OSM way and add it to layers
+2. `init_function(name)` (optional), a function to initialize Lua logic
+2. `node_function(node)`, a function to process an OSM node and add it to layers
+3. `way_function(way)`, a function to process an OSM way and add it to layers
3. `exit_function` (optional), a function to finalize Lua logic (useful to show statistics)
`node_keys` is a simple list (or in Lua parlance, a 'table') of OSM tag keys. If a node has one of those keys, it will be processed by `node_function`; if not, it'll be skipped. For example, if you wanted to show highway crossings and railway stations, it should be `{ "highway", "railway" }`. (This avoids the need to process the vast majority of nodes which contain no important tags at all.)
=====================================
include/coordinates.h
=====================================
@@ -116,20 +116,18 @@ public:
TileCoordinates index;
uint zoom;
bool hires;
+ bool endZoom;
Box clippingBox;
- TileBbox(TileCoordinates i, uint z, bool h);
+ TileBbox(TileCoordinates i, uint z, bool h, bool e);
std::pair<int,int> scaleLatpLon(double latp, double lon) const;
+ MultiPolygon scaleGeometry(MultiPolygon const &src) const;
std::pair<double, double> floorLatpLon(double latp, double lon) const;
Box getTileBox() const;
Box getExtendBox() const;
};
-// Round coordinates to integer coordinates of bbox
-// TODO: This should be self-intersection aware!!
-MultiPolygon round_coordinates(TileBbox const &bbox, MultiPolygon const &mp);
-
#endif //_COORDINATES_H
=====================================
include/geom.h
=====================================
@@ -25,6 +25,13 @@ using uint = unsigned int;
#include <boost/interprocess/managed_external_buffer.hpp>
#include <boost/interprocess/allocators/node_allocator.hpp>
+#define OSMID_TYPE_OFFSET 40
+#define OSMID_MASK ((1L<<OSMID_TYPE_OFFSET)-1)
+#define OSMID_SHAPE (0L<<OSMID_TYPE_OFFSET)
+#define OSMID_NODE (1L<<OSMID_TYPE_OFFSET)
+#define OSMID_WAY (2L<<OSMID_TYPE_OFFSET)
+#define OSMID_RELATION (3L<<OSMID_TYPE_OFFSET)
+
namespace bi = boost::interprocess;
typedef boost::geometry::model::d2::point_xy<double> Point;
=====================================
include/osm_lua_processing.h
=====================================
@@ -126,7 +126,7 @@ public:
bool CorrectGeometry(GeometryT &geom)
{
#if BOOST_VERSION >= 105800
- geom::validity_failure_type failure;
+ geom::validity_failure_type failure = geom::validity_failure_type::no_failure;
if (isRelation && !geom::is_valid(geom,failure)) {
if (verbose) std::cout << "Relation " << originalOsmID << " has " << boost_validity_error(failure) << std::endl;
} else if (isWay && !geom::is_valid(geom,failure)) {
=====================================
include/osm_store.h
=====================================
@@ -463,6 +463,7 @@ public:
void use_compact_store(bool use = true) { use_compact_nodes = use; }
void enforce_integrity(bool ei = true) { require_integrity = ei; }
+ bool integrity_enforced() { return require_integrity; }
void shapes_sort(unsigned int threadNum = 1);
void generated_sort(unsigned int threadNum = 1);
=====================================
include/output_object.h
=====================================
@@ -15,14 +15,14 @@
#include "osmformat.pb.h"
#include "vector_tile.pb.h"
-enum OutputGeometryType { POINT_, LINESTRING_, MULTILINESTRING_, POLYGON_ };
+enum OutputGeometryType : unsigned int { POINT_, LINESTRING_, MULTILINESTRING_, POLYGON_ };
#define OSMID_TYPE_OFFSET 40
-#define OSMID_MASK ((1L<<OSMID_TYPE_OFFSET)-1)
-#define OSMID_SHAPE (0L<<OSMID_TYPE_OFFSET)
-#define OSMID_NODE (1L<<OSMID_TYPE_OFFSET)
-#define OSMID_WAY (2L<<OSMID_TYPE_OFFSET)
-#define OSMID_RELATION (3L<<OSMID_TYPE_OFFSET)
+#define OSMID_MASK ((1ULL<<OSMID_TYPE_OFFSET)-1)
+#define OSMID_SHAPE (0ULL<<OSMID_TYPE_OFFSET)
+#define OSMID_NODE (1ULL<<OSMID_TYPE_OFFSET)
+#define OSMID_WAY (2ULL<<OSMID_TYPE_OFFSET)
+#define OSMID_RELATION (3ULL<<OSMID_TYPE_OFFSET)
//\brief Display the geometry type
std::ostream& operator<<(std::ostream& os, OutputGeometryType geomType);
=====================================
include/shp_mem_tiles.h
=====================================
@@ -4,6 +4,8 @@
#include "tile_data.h"
+extern bool verbose;
+
class ShpMemTiles : public TileDataSource
{
public:
@@ -29,7 +31,10 @@ public:
template <typename GeometryT>
double AreaIntersecting(const std::string &layerName, GeometryT &g) const {
auto f = indices.find(layerName);
- if (f==indices.end()) { std::cerr << "Couldn't find indexed layer " << layerName << std::endl; return false; }
+ if (f==indices.end()) {
+ if (verbose) std::cerr << "Couldn't find indexed layer " << layerName << std::endl;
+ return false;
+ }
Box box; geom::envelope(g, box);
std::vector <IndexValue> results;
f->second.query(geom::index::intersects(box), back_inserter(results));
=====================================
server/server.rb
=====================================
@@ -111,10 +111,10 @@ class MapServer
else
puts "Starting local server"
- require 'rack'
+ require 'rackup'
server = MapServer.new(ARGV[0],0)
app = Proc.new { |env| server.call(env) }
- Rack::Handler::WEBrick.run(app)
+ Rackup::Handler::WEBrick.run(app)
end
end
=====================================
src/coordinates.cpp
=====================================
@@ -72,10 +72,11 @@ void fillCoveredTiles(unordered_set<TileCoordinates> &tileSet) {
// ------------------------------------------------------
// Helper class for dealing with spherical Mercator tiles
-TileBbox::TileBbox(TileCoordinates i, uint z, bool h) {
+TileBbox::TileBbox(TileCoordinates i, uint z, bool h, bool e) {
zoom = z;
index = i;
hires = h;
+ endZoom = e;
minLon = tilex2lon(i.x ,zoom);
minLat = tiley2lat(i.y+1,zoom);
maxLon = tilex2lon(i.x+1,zoom);
@@ -96,6 +97,50 @@ pair<int,int> TileBbox::scaleLatpLon(double latp, double lon) const {
return pair<int,int>(x,y);
}
+MultiPolygon TileBbox::scaleGeometry(MultiPolygon const &src) const {
+ MultiPolygon dst;
+ for(auto poly: src) {
+ Polygon p;
+
+ // Copy the outer ring
+ Ring outer;
+ std::vector<Point> points;
+ int lastx=INT_MAX, lasty=INT_MAX;
+ for(auto &i: poly.outer()) {
+ auto scaled = scaleLatpLon(i.y(), i.x());
+ Point pt(scaled.second, scaled.first);
+ if (scaled.second!=lastx || scaled.first!=lasty) points.push_back(pt);
+ lastx=scaled.second; lasty=scaled.first;
+ }
+ if (points.size()<4) continue;
+ geom::append(outer,points);
+ geom::append(p,outer);
+
+ // Copy the inner rings
+ int num_rings = 0;
+ for(auto &r: poly.inners()) {
+ Ring inner;
+ points.clear();
+ lastx=INT_MAX, lasty=INT_MAX;
+ for(auto &i: r) {
+ auto scaled = scaleLatpLon(i.y(), i.x());
+ Point pt(scaled.second, scaled.first);
+ if (scaled.second!=lastx || scaled.first!=lasty) points.push_back(pt);
+ lastx=scaled.second; lasty=scaled.first;
+ }
+ if (points.size()<4) continue;
+ geom::append(inner,points);
+ num_rings++;
+ geom::interior_rings(p).resize(num_rings);
+ geom::append(p, inner, num_rings-1);
+ }
+
+ // Add to multipolygon
+ dst.push_back(p);
+ }
+ return dst;
+}
+
pair<double, double> TileBbox::floorLatpLon(double latp, double lon) const {
auto p = scaleLatpLon(latp, lon);
return std::make_pair( -(p.second * yscale - maxLatp), p.first * xscale + minLon);
@@ -113,28 +158,6 @@ Box TileBbox::getExtendBox() const {
geom::make<Point>( maxLon+(maxLon-minLon)*(8191.0/8192.0), maxLatp+(maxLatp-minLatp)*2.0));
}
-MultiPolygon round_coordinates(TileBbox const &bbox, MultiPolygon const &mp)
-{
- MultiPolygon combined_mp;
- for(auto new_p: mp) {
- for(auto &i: new_p.outer()) {
- auto round_i = bbox.floorLatpLon(i.y(), i.x());
- i = Point(round_i.second, round_i.first);
- }
-
- for(auto &r: new_p.inners()) {
- for(auto &i: r) {
- auto round_i = bbox.floorLatpLon(i.y(), i.x());
- i = Point(round_i.second, round_i.first);
- }
- }
-
- boost::geometry::remove_spikes(new_p);
- simplify_combine(combined_mp, std::move(new_p));
- }
- return combined_mp;
-}
-
template<typename T>
void impl_insertIntermediateTiles(T const &points, uint baseZoom, std::unordered_set<TileCoordinates> &tileSet) {
Point p2(0, 0);
=====================================
src/osm_lua_processing.cpp
=====================================
@@ -66,8 +66,9 @@ OsmLuaProcessing::OsmLuaProcessing(
// ---- Call init_function of Lua logic
- luaState("if init_function~=nil then init_function() end");
-
+ if (!!luaState["init_function"]) {
+ luaState["init_function"](this->config.projectName);
+ }
}
OsmLuaProcessing::~OsmLuaProcessing() {
@@ -414,7 +415,7 @@ void OsmLuaProcessing::LayerAsCentroid(const string &layerName) {
cout << "Couldn't find " << (isRelation ? "relation " : isWay ? "way " : "node " ) << originalOsmID << ": " << err.what() << endl;
return;
} catch (geom::centroid_exception &err) {
- cerr << "Problem geometry " << (isRelation ? "relation " : isWay ? "way " : "node " ) << originalOsmID << ": " << err.what() << endl;
+ if (verbose) cerr << "Problem geometry " << (isRelation ? "relation " : isWay ? "way " : "node " ) << originalOsmID << ": " << err.what() << endl;
return;
} catch (std::invalid_argument &err) {
cerr << "Error in OutputObjectOsmStore constructor for " << (isRelation ? "relation " : isWay ? "way " : "node " ) << originalOsmID << ": " << err.what() << endl;
=====================================
src/osm_store.cpp
=====================================
@@ -1,6 +1,7 @@
#include "osm_store.h"
#include <iostream>
+#include <fstream>
#include <iterator>
#include <unordered_map>
@@ -116,7 +117,7 @@ void mmap_dir_t::open_mmap_file(std::string const &dir_filename, size_t file_siz
std::string new_filename = mmap_dir_filename + "/mmap_" + to_string(mmap_dir.files.size()) + ".dat";
std::cout << "Filename: " << new_filename << ", size: " << mmap_file_size << std::endl;
- if(boost::filesystem::ofstream(new_filename.c_str()).fail())
+ if(std::ofstream(new_filename.c_str()).fail())
throw std::runtime_error("Failed to open mmap file");
boost::filesystem::resize_file(new_filename.c_str(), 0);
boost::filesystem::resize_file(new_filename.c_str(), mmap_file_size);
@@ -131,7 +132,7 @@ void mmap_dir_t::resize_mmap_file(size_t add_size)
auto size = increase + (add_size + alignment) - (add_size % alignment);
std::string new_filename = mmap_dir_filename + "/mmap_" + to_string(mmap_dir.files.size()) + ".dat";
- if(boost::filesystem::ofstream(new_filename.c_str()).fail())
+ if(std::ofstream(new_filename.c_str()).fail())
throw std::runtime_error("Failed to open mmap file");
boost::filesystem::resize_file(new_filename.c_str(), size);
mmap_file_thread_ptr = std::make_shared<mmap_file>(new_filename.c_str(), 0);
=====================================
src/output_object.cpp
=====================================
@@ -124,12 +124,28 @@ Geometry buildWayGeometry(OSMStore &osmStore, OutputObject const &oo, const Tile
Box box = bbox.clippingBox;
- for(auto const &p: input) {
- for(auto const &inner: p.inners()) {
- for(std::size_t i = 0; i < inner.size() - 1; ++i)
- {
- Point p1 = inner[i];
- Point p2 = inner[i + 1];
+ if (bbox.endZoom) {
+ for(auto const &p: input) {
+ for(auto const &inner: p.inners()) {
+ for(std::size_t i = 0; i < inner.size() - 1; ++i)
+ {
+ Point p1 = inner[i];
+ Point p2 = inner[i + 1];
+
+ if(geom::within(p1, bbox.clippingBox) != geom::within(p2, bbox.clippingBox)) {
+ box.min_corner() = Point(
+ std::min(box.min_corner().x(), std::min(p1.x(), p2.x())),
+ std::min(box.min_corner().y(), std::min(p1.y(), p2.y())));
+ box.max_corner() = Point(
+ std::max(box.max_corner().x(), std::max(p1.x(), p2.x())),
+ std::max(box.max_corner().y(), std::max(p1.y(), p2.y())));
+ }
+ }
+ }
+
+ for(std::size_t i = 0; i < p.outer().size() - 1; ++i) {
+ Point p1 = p.outer()[i];
+ Point p2 = p.outer()[i + 1];
if(geom::within(p1, bbox.clippingBox) != geom::within(p2, bbox.clippingBox)) {
box.min_corner() = Point(
@@ -142,35 +158,27 @@ Geometry buildWayGeometry(OSMStore &osmStore, OutputObject const &oo, const Tile
}
}
- for(std::size_t i = 0; i < p.outer().size() - 1; ++i) {
- Point p1 = p.outer()[i];
- Point p2 = p.outer()[i + 1];
-
- if(geom::within(p1, bbox.clippingBox) != geom::within(p2, bbox.clippingBox)) {
- box.min_corner() = Point(
- std::min(box.min_corner().x(), std::min(p1.x(), p2.x())),
- std::min(box.min_corner().y(), std::min(p1.y(), p2.y())));
- box.max_corner() = Point(
- std::max(box.max_corner().x(), std::max(p1.x(), p2.x())),
- std::max(box.max_corner().y(), std::max(p1.y(), p2.y())));
- }
- }
+ Box extBox = bbox.getExtendBox();
+ box.min_corner() = Point(
+ std::max(box.min_corner().x(), extBox.min_corner().x()),
+ std::max(box.min_corner().y(), extBox.min_corner().y()));
+ box.max_corner() = Point(
+ std::min(box.max_corner().x(), extBox.max_corner().x()),
+ std::min(box.max_corner().y(), extBox.max_corner().y()));
}
- Box extBox = bbox.getExtendBox();
- box.min_corner() = Point(
- std::max(box.min_corner().x(), extBox.min_corner().x()),
- std::max(box.min_corner().y(), extBox.min_corner().y()));
- box.max_corner() = Point(
- std::min(box.max_corner().x(), extBox.max_corner().x()),
- std::min(box.max_corner().y(), extBox.max_corner().y()));
-
Polygon clippingPolygon;
geom::convert(box, clippingPolygon);
try {
MultiPolygon mp;
- geom::intersection(input, clippingPolygon, mp);
+ if (geom::within(input, clippingPolygon)) { geom::assign(mp, input); return mp; }
+ // Work around boost::geometry intersection issues by doing each constituent polygon at a time
+ for (auto &p : input) {
+ MultiPolygon out;
+ geom::intersection(p, clippingPolygon, out);
+ for (auto &o : out) mp.emplace_back(move(o));
+ }
return mp;
} catch (geom::overlay_invalid_input_exception &err) {
std::cout << "Couldn't clip polygon (self-intersection)" << std::endl;
=====================================
src/read_pbf.cpp
=====================================
@@ -90,7 +90,11 @@ bool PbfReader::ReadWays(OsmLuaProcessing &output, PrimitiveGroup &pg, Primitive
int64_t nodeId = 0;
for (int k=0; k<pbfWay.refs_size(); k++) {
nodeId += pbfWay.refs(k);
- llVec.push_back(osmStore.nodes_at(static_cast<NodeID>(nodeId)));
+ try {
+ llVec.push_back(osmStore.nodes_at(static_cast<NodeID>(nodeId)));
+ } catch (std::out_of_range &err) {
+ if (osmStore.integrity_enforced()) throw err;
+ }
}
}
=====================================
src/shared_data.cpp
=====================================
@@ -41,7 +41,8 @@ uint LayerDefinition::addLayer(string name, uint minzoom, uint maxzoom,
layerOrder.push_back(r);
} else {
if (layerMap.count(writeTo) == 0) {
- throw out_of_range("ERROR: addLayer(): the layer to write, named as \"" + writeTo + "\", doesn't exist.");
+ cerr << "ERROR: addLayer(): the layer to write, named as \"" + writeTo + "\", doesn't exist." << endl;
+ exit (EXIT_FAILURE);
}
uint lookingFor = layerMap[writeTo];
for (auto it = layerOrder.begin(); it!= layerOrder.end(); ++it) {
=====================================
src/shp_mem_tiles.cpp
=====================================
@@ -2,6 +2,7 @@
#include <iostream>
using namespace std;
namespace geom = boost::geometry;
+extern bool verbose;
ShpMemTiles::ShpMemTiles(OSMStore &osmStore, uint baseZoom)
: TileDataSource(baseZoom), osmStore(osmStore)
@@ -20,7 +21,7 @@ vector<uint> ShpMemTiles::QueryMatchingGeometries(const string &layerName, bool
// Find the layer
auto f = indices.find(layerName); // f is an RTree
if (f==indices.end()) {
- cerr << "Couldn't find indexed layer " << layerName << endl;
+ if (verbose) cerr << "Couldn't find indexed layer " << layerName << endl;
return vector<uint>(); // empty, relations not supported
}
=====================================
src/tile_worker.cpp
=====================================
@@ -122,6 +122,18 @@ void CheckNextObjectAndMerge(OSMStore &osmStore, OutputObjectsConstIt &jt, Outpu
}
}
+void RemoveInnersBelowSize(MultiPolygon &g, double filterArea) {
+ for (auto &outer : g) {
+ outer.inners().erase(std::remove_if(
+ outer.inners().begin(),
+ outer.inners().end(),
+ [&](const Ring &inner) -> bool {
+ return std::fabs(geom::area(inner)) < filterArea;
+ }),
+ outer.inners().end());
+ }
+}
+
void ProcessObjects(OSMStore &osmStore, OutputObjectsConstIt ooSameLayerBegin, OutputObjectsConstIt ooSameLayerEnd,
class SharedData &sharedData, double simplifyLevel, double filterArea, bool combinePolygons, unsigned zoom, const TileBbox &bbox,
vector_tile::Tile_Layer *vtLayer, vector<string> &keyList, vector<vector_tile::Tile_Value> &valueList) {
@@ -140,7 +152,7 @@ void ProcessObjects(OSMStore &osmStore, OutputObjectsConstIt ooSameLayerBegin, O
featurePtr->set_type(vector_tile::Tile_GeomType_POINT);
oo->writeAttributes(&keyList, &valueList, featurePtr, zoom);
- if (sharedData.config.includeID) { featurePtr->set_id(oo->objectID); }
+ if (sharedData.config.includeID) { featurePtr->set_id(oo->objectID & OSMID_MASK); }
} else {
Geometry g;
try {
@@ -152,6 +164,7 @@ void ProcessObjects(OSMStore &osmStore, OutputObjectsConstIt ooSameLayerBegin, O
if (oo->geomType == POLYGON_ && filterArea > 0.0) {
if (geom::area(g)<filterArea) continue;
+ RemoveInnersBelowSize(boost::get<MultiPolygon>(g), filterArea);
}
//This may increment the jt iterator
@@ -171,7 +184,7 @@ void ProcessObjects(OSMStore &osmStore, OutputObjectsConstIt ooSameLayerBegin, O
boost::apply_visitor(w, g);
if (featurePtr->geometry_size()==0) { vtLayer->mutable_features()->RemoveLast(); continue; }
oo->writeAttributes(&keyList, &valueList, featurePtr, zoom);
- if (sharedData.config.includeID) { featurePtr->set_id(oo->objectID); }
+ if (sharedData.config.includeID) { featurePtr->set_id(oo->objectID & OSMID_MASK); }
}
}
@@ -259,7 +272,7 @@ bool outputProc(boost::asio::thread_pool &pool, SharedData &sharedData, OSMStore
{
// Create tile
vector_tile::Tile tile;
- TileBbox bbox(coordinates, zoom, sharedData.config.highResolution && zoom==sharedData.config.endZoom);
+ TileBbox bbox(coordinates, zoom, sharedData.config.highResolution && zoom==sharedData.config.endZoom, zoom==sharedData.config.endZoom);
if (sharedData.config.clippingBoxFromJSON && (sharedData.config.maxLon<=bbox.minLon
|| sharedData.config.minLon>=bbox.maxLon || sharedData.config.maxLat<=bbox.minLat
|| sharedData.config.minLat>=bbox.maxLat)) { return true; }
=====================================
src/tilemaker.cpp
=====================================
@@ -188,12 +188,12 @@ int main(int argc, char* argv[]) {
po::positional_options_description p;
p.add("input", -1);
po::variables_map vm;
- try {
- po::store(po::command_line_parser(argc, argv).options(desc).positional(p).run(), vm);
- } catch (const po::unknown_option& ex) {
- cerr << "Unknown option: " << ex.get_option_name() << endl;
- return -1;
- }
+ try {
+ po::store(po::command_line_parser(argc, argv).options(desc).positional(p).run(), vm);
+ } catch (const po::unknown_option& ex) {
+ cerr << "Unknown option: " << ex.get_option_name() << endl;
+ return -1;
+ }
po::notify(vm);
if (vm.count("help")) { cout << desc << endl; return 0; }
@@ -342,7 +342,7 @@ int main(int argc, char* argv[]) {
});
if (ret != 0) return ret;
}
- osmStore.clear(); // no longer needed
+ void_mmap_allocator::shutdown(); // this clears the mmap'ed nodes/ways/relations (quickly!)
}
// ---- Initialise SharedData
@@ -448,7 +448,7 @@ int main(int argc, char* argv[]) {
}
if (hasClippingBox) {
- if(!boost::geometry::intersects(TileBbox(it, zoom, false).getTileBox(), clippingBox))
+ if(!boost::geometry::intersects(TileBbox(it, zoom, false, false).getTileBox(), clippingBox))
continue;
}
@@ -502,8 +502,6 @@ int main(int argc, char* argv[]) {
}
#endif
- void_mmap_allocator::shutdown();
-
cout << endl << "Filled the tileset with good things at " << sharedData.outputFile << endl;
}
=====================================
src/write_geometry.cpp
=====================================
@@ -28,12 +28,9 @@ void WriteGeometryVisitor::operator()(const Point &p) const {
// Multipolygon
void WriteGeometryVisitor::operator()(const MultiPolygon &mp) const {
- MultiPolygon current;
- if (simplifyLevel>0) {
- current = simplify(round_coordinates(*bboxPtr, mp), simplifyLevel);
- } else {
- current = mp;
- }
+ MultiPolygon current = bboxPtr->scaleGeometry(mp);
+ if (simplifyLevel>0) { current = simplify(current, simplifyLevel/bboxPtr->xscale); geom::remove_spikes(current); }
+ if (geom::is_empty(current)) return;
#if BOOST_VERSION >= 105800
geom::validity_failure_type failure;
@@ -53,24 +50,23 @@ void WriteGeometryVisitor::operator()(const MultiPolygon &mp) const {
pair<int,int> lastPos(0,0);
for (MultiPolygon::const_iterator it = current.begin(); it != current.end(); ++it) {
- XYString scaledString;
+ XYString points;
Ring ring = geom::exterior_ring(*it);
for (auto jt = ring.begin(); jt != ring.end(); ++jt) {
- pair<int,int> xy = bboxPtr->scaleLatpLon(jt->get<1>(), jt->get<0>());
- scaledString.push_back(xy);
+ pair<int,int> xy(jt->get<1>(), jt->get<0>());
+ points.push_back(xy);
}
- bool success = writeDeltaString(&scaledString, featurePtr, &lastPos, true);
+ bool success = writeDeltaString(&points, featurePtr, &lastPos, true);
if (!success) continue;
InteriorRing interiors = geom::interior_rings(*it);
for (auto ii = interiors.begin(); ii != interiors.end(); ++ii) {
- scaledString.clear();
- XYString scaledInterior;
+ points.clear();
for (auto jt = ii->begin(); jt != ii->end(); ++jt) {
- pair<int,int> xy = bboxPtr->scaleLatpLon(jt->get<1>(), jt->get<0>());
- scaledString.push_back(xy);
+ pair<int,int> xy(jt->get<1>(), jt->get<0>());
+ points.push_back(xy);
}
- writeDeltaString(&scaledString, featurePtr, &lastPos, true);
+ writeDeltaString(&points, featurePtr, &lastPos, true);
}
}
featurePtr->set_type(vector_tile::Tile_GeomType_POLYGON);
View it on GitLab: https://salsa.debian.org/debian-gis-team/tilemaker/-/compare/f058525647d6c4fabab9ca203ffce8949ba9897d...3310c4398e1ac3ac575eab0aa96c383db80335d9
--
View it on GitLab: https://salsa.debian.org/debian-gis-team/tilemaker/-/compare/f058525647d6c4fabab9ca203ffce8949ba9897d...3310c4398e1ac3ac575eab0aa96c383db80335d9
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20230309/5ef8c049/attachment-0001.htm>
More information about the Pkg-grass-devel
mailing list