[Git][debian-gis-team/mkgmap-splitter][upstream] New upstream version 0.0.0+svn591
Bas Couwenberg
gitlab at salsa.debian.org
Sun Apr 1 09:57:56 UTC 2018
Bas Couwenberg pushed to branch upstream at Debian GIS Project / mkgmap-splitter
Commits:
9be5c981 by Bas Couwenberg at 2018-04-01T11:21:34+02:00
New upstream version 0.0.0+svn591
- - - - -
9 changed files:
- doc/splitter.1
- doc/splitter.1.xml
- doc/splitter.txt
- resources/splitter-version.properties
- src/uk/me/parabola/splitter/Main.java
- src/uk/me/parabola/splitter/Utils.java
- src/uk/me/parabola/splitter/args/SplitterParams.java
- src/uk/me/parabola/splitter/tools/BitReader.java
- src/uk/me/parabola/splitter/tools/BitWriter.java
Changes:
=====================================
doc/splitter.1
=====================================
--- a/doc/splitter.1
+++ b/doc/splitter.1
@@ -36,20 +36,15 @@ Tiles join exactly with no overlap or gaps.
You will need a lot of memory on your computer if you intend to split a
large area.
A few options allow configuring how much memory you need.
-With the default parameters, you need about 4-5 bytes for every node and
+With the default parameters, you need about 2 bytes for every node and
way.
-This doesn't sound a lot but there are about 1700 million nodes in the
-whole planet file and so you cannot process the whole planet in one pass
-file on a 32 bit machine using this utility as the maximum java heap
+This doesn't sound a lot but there are about 4300 million nodes in the
+whole planet file (Jan 2018) and so you cannot process the whole planet in one
+pass on a 32 bit machine using this utility, as the maximum java heap
space is 2G.
-It is possible with 64 bit java and about 7GB of heap or with multiple
+It is possible with 64 bit java and about 10GB of heap or with multiple
passes.
.PP
-The Europe extract from Cloudmade or Geofabrik can be processed within
-the 2G limit if you have sufficient memory.
-With the default options europe is split into about 750 tiles.
-The Europe extract is about half of the size of the complete planet file.
-.PP
On the other hand a single country, even a well mapped one such as
Germany or the UK, will be possible on a modest machine, even a netbook.
.SH USAGE
@@ -79,14 +74,13 @@ There are also other files produced:
The \*(T<\fItemplate.args\fR\*(T> file is a file that can
be used with the \*(T<\fB\-c\fR\*(T> option of
\fBmkgmap\fR that will compile all the files.
-You can use it as is or you can copy it and edit it to include
-your own options.
+You can use it as is or you can edit it to include your own options.
For example instead of each description being "OSM Map" it could
be "NW Scotland" as appropriate.
.PP
The \*(T<\fIareas.list\fR\*(T> file is the list of bounding
boxes that were calculated.
-If you want you can use this on a subsequent call the the
+If you want, you can use this on subsequent calls of
splitter using the \*(T<\fB\-\-split\-file\fR\*(T> option to use
exactly the same areas as last time.
This might be useful if you produce a map regularly and want to
@@ -94,8 +88,8 @@ keep the tile areas the same from month to month.
It is also useful to avoid the time it takes to regenerate the
file each time (currently about a third of the overall time
taken to perform the split).
-Of course if the map grows enough that one of the tiles overflows
-you will have to re-calculate the areas again.
+Of course if the map grows enough that one of the tiles overflows, you
+will have to re-calculate the areas again.
.PP
The \*(T<\fIareas.poly\fR\*(T> file contains the bounding
polygon of the calculated areas.
@@ -265,9 +259,9 @@ If the input file contains a bounding box, this is applied to the
grid so that nodes outside of the bounding box are ignored.
Next, if specified, the bounding polygon is used to zero those
grid elements outside of the bounding polygon area.
-If the polygon area(s) describe(s) a rectilinear area with no more
+If the polygon file describes one or more rectilinear areas with no more
than 40 vertices, \fBmkgmap-splitter\fR will try to
-create output files that fit exactly into the area, otherwise it
+create output files that fit exactly into each area, otherwise it
will approximate the polygon area with rectangles.
.TP
\*(T<\fB\-\-precomp\-sea=\fR\*(T>\fIpath\fR
=====================================
doc/splitter.1.xml
=====================================
--- a/doc/splitter.1.xml
+++ b/doc/splitter.1.xml
@@ -47,22 +47,16 @@
You will need a lot of memory on your computer if you intend to split a
large area.
A few options allow configuring how much memory you need.
- With the default parameters, you need about 4-5 bytes for every node and
+ With the default parameters, you need about 2 bytes for every node and
way.
- This doesn't sound a lot but there are about 1700 million nodes in the
- whole planet file and so you cannot process the whole planet in one pass
- file on a 32 bit machine using this utility as the maximum java heap
+ This doesn't sound a lot but there are about 4300 million nodes in the
+ whole planet file (Jan 2018) and so you cannot process the whole planet in
+ one pass on a 32 bit machine using this utility as the maximum java heap
space is 2G.
- It is possible with 64 bit java and about 7GB of heap or with multiple
+ It is possible with 64 bit java and about 10GB of heap or with multiple
passes.
</para>
<para>
- The Europe extract from Cloudmade or Geofabrik can be processed within
- the 2G limit if you have sufficient memory.
- With the default options europe is split into about 750 tiles.
- The Europe extract is about half of the size of the complete planet file.
- </para>
- <para>
On the other hand a single country, even a well mapped one such as
Germany or the UK, will be possible on a modest machine, even a netbook.
</para>
@@ -97,7 +91,7 @@ JAVA_OPTS="<replaceable>-Xmx512m</replaceable>" <command>mkgmap-splitter</comman
The <filename>template.args</filename> file is a file that can
be used with the <option>-c</option> option of
<command>mkgmap</command> that will compile all the files.
- You can use it as is or you can copy it and edit it to include
+ You can use it as is or you can edit it to include
your own options.
For example instead of each description being "OSM Map" it could
be "NW Scotland" as appropriate.
@@ -105,7 +99,7 @@ JAVA_OPTS="<replaceable>-Xmx512m</replaceable>" <command>mkgmap-splitter</comman
<para>
The <filename>areas.list</filename> file is the list of bounding
boxes that were calculated.
- If you want you can use this on a subsequent call the the
+ If you want, you can use this on subsequent calls of
splitter using the <option>--split-file</option> option to use
exactly the same areas as last time.
This might be useful if you produce a map regularly and want to
@@ -113,7 +107,7 @@ JAVA_OPTS="<replaceable>-Xmx512m</replaceable>" <command>mkgmap-splitter</comman
It is also useful to avoid the time it takes to regenerate the
file each time (currently about a third of the overall time
taken to perform the split).
- Of course if the map grows enough that one of the tiles overflows
+ Of course if the map grows enough that one of the tiles overflows,
you will have to re-calculate the areas again.
</para>
<para>
@@ -400,9 +394,9 @@ JAVA_OPTS="<replaceable>-Xmx512m</replaceable>" <command>mkgmap-splitter</comman
grid so that nodes outside of the bounding box are ignored.
Next, if specified, the bounding polygon is used to zero those
grid elements outside of the bounding polygon area.
- If the polygon area(s) describe(s) a rectilinear area with no more
+ If the polygon-file describes one or more rectilinear areas with no more
than 40 vertices, <command>mkgmap-splitter</command> will try to
- create output files that fit exactly into the area, otherwise it
+ create output files that fit exactly into each area, otherwise it
will approximate the polygon area with rectangles.
</para>
</listitem>
=====================================
doc/splitter.txt
=====================================
--- a/doc/splitter.txt
+++ b/doc/splitter.txt
@@ -19,16 +19,11 @@ The two most important features are:
You will need a lot of memory on your computer if you intend to split a large
area. A few options allow configuring how much memory you need. With the
-default parameters, you need about 4-5 bytes for every node and way. This
-doesn't sound a lot but there are about 1700 million nodes in the whole planet
-file and so you cannot process the whole planet in one pass file on a 32 bit
-machine using this utility as the maximum java heap space is 2G. It is
-possible with 64 bit java and about 7GB of heap or with multiple passes.
-
-The Europe extract from Cloudmade or Geofabrik can be processed within the 2G
-limit if you have sufficient memory. With the default options europe is split
-into about 750 tiles. The Europe extract is about half of the size of the
-complete planet file.
+default parameters, you need about 2 bytes for every node and way. This
+doesn't sound a lot but there are about 4300 million nodes in the whole planet
+file (Jan 2018) and so you cannot process the whole planet in one pass on
+a 32 bit machine using this utility, as the maximum java heap space is 2G. It is
+possible with 64 bit java and about 10GB of heap or with multiple passes.
On the other hand a single country, even a well mapped one such as Germany or
the UK, will be possible on a modest machine, even a netbook.
@@ -50,21 +45,20 @@ This will produce a number of .osm.pbf files that can be read by mkgmap.
There are also other files produced:
The ''template.args'' file is a file that can be used with the -c option of
-mkgmap that will compile all the files. You can use it as is or you can copy
-it and edit it to include your own options. For example instead of each
+mkgmap that will compile all the files. You can use it as is or you can
+edit it to include your own options. For example instead of each
description being "OSM Map" it could be "NW Scotland" as appropriate.
The ''areas.list'' file is the list of bounding boxes that were calculated. If
-you want you can use this on a subsequent call the the splitter using the
+you want, you can use this on subsequent calls of splitter using the
--split-file option to use exactly the same areas as last time. This might be
useful if you produce a map regularly and want to keep the tile areas the same
from month to month. It is also useful to avoid the time it takes to
regenerate the file each time (currently about a third of the overall time
taken to perform the split). Of course if the map grows enough that one of the
-tiles overflows you will have to re-calculate the areas again.
+tiles overflows, you will have to re-calculate the areas again.
The ''areas.poly'' file contains the bounding polygon of the calculated areas.
-See option --polygon-file how this can be used.
The ''densities-out.txt'' file is written when no split-file is given and
contains debugging information only.
@@ -167,14 +161,14 @@ will be printed because this combination rarely makes sense.
;--polygon-file
: The name of a file containing a bounding polygon in the
[http://wiki.openstreetmap.org/wiki/Osmosis/Polygon_Filter_File_Format osmosis polygon file format].
-Splitter uses this fie when calculating the areas. It first calculates a grid
+Splitter uses this file when calculating the areas. It first calculates a grid
using the given --resolution. The input file is read and for each node, a
counter is increased for the related grid area. If the input file contains a
bounding box, this is applied to the grid so that nodes outside of the
bounding box are ignored. Next, if specified, the bounding polygon is used to
-zero those grid elements outside of the bounding polygon area. If the polygon
-area(s) describe(s) a rectilinear area with no more than 40 vertices, splitter
-will try to create output files that fit exactly into the area, otherwise it
+zero those grid elements outside of the bounding polygon area. If the polygon-file
+describes one or more rectilinear areas with no more than 40 vertices, splitter
+will try to create output files that fit exactly into each area, otherwise it
will approximate the polygon area with rectangles.
;--precomp-sea
=====================================
resources/splitter-version.properties
=====================================
--- a/resources/splitter-version.properties
+++ b/resources/splitter-version.properties
@@ -1,2 +1,2 @@
-svn.version: 583
-build.timestamp: 2017-04-01T15:52:04+0100
+svn.version: 591
+build.timestamp: 2018-03-09T14:31:20+0000
=====================================
src/uk/me/parabola/splitter/Main.java
=====================================
--- a/src/uk/me/parabola/splitter/Main.java
+++ b/src/uk/me/parabola/splitter/Main.java
@@ -15,6 +15,8 @@ package uk.me.parabola.splitter;
import java.io.File;
import java.io.IOException;
+import java.time.Duration;
+import java.time.Instant;
import java.util.Arrays;
import java.util.Date;
import java.util.List;
@@ -101,7 +103,7 @@ public class Main {
healthMonitor.start();
}
- long start = System.currentTimeMillis();
+ Instant start = Instant.now();
System.out.println("Time started: " + new Date());
try {
// configure the input file handler
@@ -149,7 +151,20 @@ public class Main {
return 1;
}
System.out.println("Time finished: " + new Date());
- System.out.println("Total time taken: " + (System.currentTimeMillis() - start) / 1000 + 's');
+ Duration duration = Duration.between(start, Instant.now());
+ long seconds = duration.getSeconds();
+ if (seconds > 0) {
+ long hours = seconds / 3600;
+ seconds -= hours * 3600;
+ long minutes = seconds / 60;
+ seconds -= minutes * 60;
+ System.out.println("Total time taken: " +
+ (hours > 0 ? hours + (hours > 1 ? " hours " : " hour ") : "") +
+ (minutes > 0 ? minutes + (minutes > 1 ? " minutes " : " minute ") : "") +
+ (seconds > 0 ? seconds + (seconds > 1 ? " seconds" : " second") : ""));
+ }
+ else
+ System.out.println("Total time taken: " + duration.getNano() / 1000000 + " ms");
return rc;
}
=====================================
src/uk/me/parabola/splitter/Utils.java
=====================================
--- a/src/uk/me/parabola/splitter/Utils.java
+++ b/src/uk/me/parabola/splitter/Utils.java
@@ -69,7 +69,7 @@ public class Utils {
* @return An integer value in map units.
*/
public static int toMapUnit(double l) {
- double DELTA = 0.000001; // TODO check if we really mean this
+ double DELTA = 360.0D / (1 << 24) / 2; //Correct rounding
if (l > 0)
return (int) ((l + DELTA) * (1 << 24)/360);
return (int) ((l - DELTA) * (1 << 24)/360);
=====================================
src/uk/me/parabola/splitter/args/SplitterParams.java
=====================================
--- a/src/uk/me/parabola/splitter/args/SplitterParams.java
+++ b/src/uk/me/parabola/splitter/args/SplitterParams.java
@@ -108,7 +108,7 @@ public interface SplitterParams {
+ "Used to filter multipolygon and boundary relations for problem-list processing.")
String getBoundaryTags();
- @Option(defaultValue="5", description = "The lowest admin_level value that should be kept complete. Reasonable values are 2 .. 11."
+ @Option(defaultValue="5", description = "The lowest admin_level value that should be kept complete. Reasonable values are 2 .. 11. "
+ "Used to filter boundary relations for problem-list processing. Ignored when keep-complete is false.")
int getWantedAdminLevel();
=====================================
src/uk/me/parabola/splitter/tools/BitReader.java
=====================================
--- a/src/uk/me/parabola/splitter/tools/BitReader.java
+++ b/src/uk/me/parabola/splitter/tools/BitReader.java
@@ -1,23 +1,19 @@
/*
- * Copyright (C) 2008 Steve Ratcliffe
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- *
- * Author: Steve Ratcliffe
- * Create date: 29-Aug-2008
+ * Copyright (C) 2017, Gerd Petermann
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 3 or
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
*/
package uk.me.parabola.splitter.tools;
/**
- * Read an array as a bit stream.
+ * Read an array as a bit stream. Based on code in mkgmap.
*
* @author Steve Ratcliffe
* @author Gerd Petermann
=====================================
src/uk/me/parabola/splitter/tools/BitWriter.java
=====================================
--- a/src/uk/me/parabola/splitter/tools/BitWriter.java
+++ b/src/uk/me/parabola/splitter/tools/BitWriter.java
@@ -1,27 +1,24 @@
/*
- * Copyright (C) 2006 Steve Ratcliffe
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- *
- * Author: Steve Ratcliffe
- * Create date: 14-Dec-2006
+ * Copyright (C) 2017, Gerd Petermann
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 3 or
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
*/
package uk.me.parabola.splitter.tools;
import java.util.Arrays;
/**
- * A class to write the bitstream.
+ * A class to write the bitstream. Based on code in mkgmap.
*
* @author Steve Ratcliffe
+ * @author Gerd Petermann
*/
public class BitWriter {
// Choose so that chunks will not fill it.
View it on GitLab: https://salsa.debian.org/debian-gis-team/mkgmap-splitter/commit/9be5c981d928b470fa36e6312897975aa37151c5
---
View it on GitLab: https://salsa.debian.org/debian-gis-team/mkgmap-splitter/commit/9be5c981d928b470fa36e6312897975aa37151c5
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-grass-devel/attachments/20180401/e5f1cc49/attachment-0001.html>
More information about the Pkg-grass-devel
mailing list