Fwd: Re: change FILESORT_VARINT to 32-bits

[Rainer Dorsch]

Hello everybody,

I am impressed that you already packaged routino 3.4, just hours after the 
release. Many thanks for that.

I see that routino 3.4 contains an important bugfix for its users. According to 
Andrew serveral users from Germany contacted him for this issue. Is there a 
chance to get 3.4 into backports for bookworm?

Thanks again
Rainer

----------  Weitergeleitete Nachricht  ----------

Betreff: Re: change FILESORT_VARINT to 32-bits
Datum: Sonntag, 11. Juni 2023, 17:55:33 CEST
Von: amb <amb at routino.org>
An: Rainer Dorsch <ml at bokomoko.de>

Hi,

> many thanks for sharing routino, I use it very frequently within qmapshack 
for 
> planning bike tours. In particular the interactive routing works very 
> impressively within qmapshack.
>
> I update from time to time the osm database for routing. In the last update 
> for Germany, I hint an issue in planetsplitter
>
> Routino Fatal Error (relationsx.c:240): Route relation contains too much 
data 
> (change FILESORT_VARINT to 32-bits?)
>
> After I opened a bug report at qmapshack (I thought it might be that 
qmapshack 
> is using routino in the wrong way, since I also upgraded Debian to Debian 
12):
>
> https://github.com/Maproom/qmapshack/issues/604
>
> But it seems that others have hit the same issue. The error message seems to 
> give a hint " change FILESORT_VARINT to 32-bits?". I am wondering if that is 
> already a compile time switch or a major rework of the code.

The change suggested by the error message is just a one-line change in
the file sorting.h but it will make the data processing less efficient
in some cases which is why it does not have that value by default.

The problem is caused by a route relation that has more than ~8000
nodes and ways.  It is not forbidden to have a relation with so many
members but the maximum size of a relation is *recommended* to be no
more than 300 items (https://wiki.openstreetmap.org/wiki/Relation#Size)
so these are exceptional relations.


> I also saw in the svn repo that you are about to release 3.4, I am wondering 
> if it is fixed in there.

Yes, your question is well timed, I have just (less than an hour ago)
released version 3.4 and the problem is fixed there.

-------------------- ChangeLog --------------------
2022-07-10 [r2110]  Andrew M. Bishop <amb>

	* src/relationsx.c, src/waysx.c: Drop data and log an error rather
	  than exit if a way or relation contains too much data.
-------------------- ChangeLog --------------------

The fix that I used is perhaps not the optimum one but when there is
too much data (a route relation with more than about 8000 members)
Routino now discards the extra data.  This is a rare case and route
relations only have a small effect on the routing database.  Dropping
a few route members seems better than making the sorting less
efficient for everything else.

-- 
Andrew.
----------------------------------------------------------------------
Andrew M. Bishop                               http://www.routino.org/

-------------------------------------------------------------
-- 
Rainer Dorsch
http://bokomoko.de/