Bug#1087811: josm: MapWithAI plugin – java.lang.ClassNotFoundException: org.apache.commons.io.input.BoundedInputStream

Mon Nov 18 23:15:50 GMT 2024

Package: josm
Version: 0.0.svn19253+dfsg-1
Severity: normal
X-Debbugs-Cc: bugs.debian.org at knackich.de

Dear Maintainer,

I’m trying to use the MapWithAI plugin with a specific data source and 
it fails to download data with this error:

java.lang.ClassNotFoundException: 
org.apache.commons.io.input.BoundedInputStream

The problem was introduced at some point during October 2024. I’ve used 
the plugin successfully many times at least until September 2024.

Steps to reproduce:

Add the plugin. Open the Preferences (F12) › Plugins. Search for 
"mapwithai". Check the entry and confirm with OK.

Add a custom data source: Menu › Data › MapWithAI › MapWithAI 
Preferences. In the bottom part, press the "+" button to add a custom 
data source. In the dialog that comes up, enter:

2. Enter Service URL = 
https://hfs.github.io/brandenburg-addresses/missing.pmtiles

3. Enter name for this source = GeoBasis-DE/LGB (2024): Georeferenzierte 
Adresse

4. What is the type of this source? = PMTILES

Close the preferences with OK.

Now download some OpenStreetMap data. File › Download data…

Zoom in on any region in the map – the location in the world doesn’t 
matter – and draw a small rectangle › Download.

The OpenStreetMap data is downloaded and shown as new editor layer.

Now try to download extra data using the plugin. Data › MapWithAI › 
GeoBasis-DE/LGB (2024): Georeferenzierte Adresse

The expected behavior would be that data gets fetched for the selected 
location and displayed in a new layer called "MapWithAI". If the 
selected location happens to be in Brandenburg, Germany, there might be 
actual data to download. Anywhere else, there will be no data and the 
"MapWithAI" layer would be empty.

Instead, the error dialog comes up. This Java exception is logged:

2024-11-18 21:56:19.278 SEVERE: Handled by bug report queue: 
java.lang.NoClassDefFoundError. Cause: java.lang.NoClassDefFoundError: 
org/apache/commons/io/input/BoundedInputStream. Cause: 
java.lang.ClassNotFoundException: 
org.apache.commons.io.input.BoundedInputStream
java.lang.NoClassDefFoundError
	at 
java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
	at 
java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
	at 
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
	at 
java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:542)
	at 
java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:567)
	at java.base/java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:653)
	at 
org.openstreetmap.josm.plugins.mapwithai.actions.AddMapWithAILayerAction.realRun(AddMapWithAILayerAction.java:123)
	at 
org.openstreetmap.josm.plugins.mapwithai.actions.AddMapWithAILayerAction.lambda$actionPerformed$2(AddMapWithAILayerAction.java:88)
	at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.NoClassDefFoundError: 
org/apache/commons/io/input/BoundedInputStream
	at 
org.openstreetmap.josm.plugins.pmtiles.lib.PMTiles.decompressInputStream(PMTiles.java:263)
	at 
org.openstreetmap.josm.plugins.pmtiles.lib.PMTiles.readRootDirectory(PMTiles.java:69)
	at 
org.openstreetmap.josm.plugins.mapwithai.backend.BoundingBoxMapWithAIDownloader.readMvt(BoundingBoxMapWithAIDownloader.java:296)
	at 
org.openstreetmap.josm.plugins.mapwithai.backend.BoundingBoxMapWithAIDownloader.parseDataSet(BoundingBoxMapWithAIDownloader.java:265)
	at 
org.openstreetmap.josm.io.BoundingBoxDownloader.parseOsm(BoundingBoxDownloader.java:217)
	at 
org.openstreetmap.josm.plugins.mapwithai.backend.BoundingBoxMapWithAIDownloader.parseOsm(BoundingBoxMapWithAIDownloader.java:160)
	at 
org.openstreetmap.josm.plugins.mapwithai.backend.MapWithAIDataUtils.lambda$download$6(MapWithAIDataUtils.java:178)
	at 
java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1456)
	at 
java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at 
java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
	at 
java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
	at 
java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
Caused by: java.lang.ClassNotFoundException: 
org.apache.commons.io.input.BoundedInputStream
	at 
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at 
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
	... 13 more

This error affects only the Debian version of JOSM. When I download the 
official release from https://josm.openstreetmap.de/josm-tested.jar and 
run it with

java --add-exports=java.base/sun.security.action=ALL-UNNAMED 
--add-exports=java.desktop/com.sun.imageio.plugins.jpeg=ALL-UNNAMED 
--add-exports=java.desktop/com.sun.imageio.spi=ALL-UNNAMED -jar 
josm-tested.jar

it works as expected. The official release version is currently 19253, 
which is the same as used in the Debian package.

I believe this is a dependency version conflict regarding the Apache 
commons-io library.

The MapWithAI plugin (version 837) gets installed into
~/.local/share/JOSM/plugins/mapwithai.jar

It pulls in other plugin as dependency: apache-commons (version 36349) in
~/.local/share/JOSM/plugins/apache-commons.jar
and pmtiles (version 36219) in
~/.local/share/JOSM/plugins/pmtiles.jar

PMTiles.decompressInputStream(PMTiles.java:263) from the stacktrace is 
this place in the PMTiles plugin:

https://josm.openstreetmap.de/browser/osm/applications/editors/josm/plugins/pmtiles/src/main/java/org/openstreetmap/josm/plugins/pmtiles/lib/PMTiles.java?rev=36219#L263

The line that fails is

new GzipCompressorInputStream(inputStream);

org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream 
comes from commons-compress.

According to 
https://josm.openstreetmap.de/browser/josm/trunk/nodist/pom.xml?rev=19251#L287 
JOSM uses version 1.27.1 of commons-compress.

The constructor of GzipCompressorInputStream,

https://github.com/apache/commons-compress/blob/rel/commons-compress-1.27.1/src/main/java/org/apache/commons/compress/compressors/gzip/GzipCompressorInputStream.java#L155

calls

countingStream = 
BoundedInputStream.builder().setInputStream(inputStream).get();

org.apache.commons.io.input.BoundedInputStream is a dependency from 
commons-io and the class the ClassNotFoundException was about.

BoundedInputStream is indeed included in the apache-commons plugin:

$ unzip -l ~/.local/share/JOSM/plugins/apache-commons.jar | grep 
BoundedInputStream 
                                 (base)
       732  2024-08-16 20:14 
org/apache/commons/compress/utils/BoundedInputStream.class
      1925  2024-09-15 14:44 
org/apache/commons/io/input/BoundedInputStream$AbstractBuilder.class
      1532  2024-09-15 14:44 
org/apache/commons/io/input/BoundedInputStream$Builder.class
      3831  2024-09-15 14:44 
org/apache/commons/io/input/BoundedInputStream.class

By running JOSM with -verbose:class, I noticed that 
GzipCompressorInputStream gets loaded from the system-wide version and 
not from the plugin:

[28,500s][info][class,load] 
org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream 
source: file:/usr/share/java/commons-compress.jar

In contrast, when running JOSM’s official release, it gets loaded from 
the plugin

[25,273s][info][class,load] 
org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream 
source: file:/home/user/.local/share/JOSM/plugins/apache-commons.jar

I also found it noteworthy that BoundedInputStream is contained in the 
official JOSM JAR, but not Debian’s JAR?

[27,222s][info][class,load] 
org.apache.commons.io.input.BoundedInputStream source: 
file:/home/user/Download/josm-tested.jar
[27,230s][info][class,load] 
org.apache.commons.io.input.BoundedInputStream$AbstractBuilder source: 
file:/home/user/Download/josm-tested.jar
[27,230s][info][class,load] 
org.apache.commons.io.input.BoundedInputStream$Builder source: 
file:/home/user/Download/josm-tested.jar

$ unzip -l /usr/share/josm/josm-0.0.svn19253+dfsg.jar | grep BoundedInput
      1862  2024-11-01 05:02 
org/openstreetmap/josm/io/OsmPbfReader$BoundedInputStream.class

I have no idea how the class loaders are set up to support plugins. 
Could it be that GzipCompressorInputStream from 
/usr/share/java/commons-compress.jar can’t "see" BoundedInputStream from 
the plugin, because that would require to go through a different class 
loader?

As a little experiment I edited 
/usr/share/josm/josm-0.0.svn19253+dfsg.jar and added 
/usr/share/java/commons-io.jar to the Class-Path: line in 
META-INF/MANIFEST.MF. This fixed the immediate issue, but I have no idea 
about possible side effects.

Thanks!
Hermann

-- System Information:
Debian Release: trixie/sid
   APT prefers testing
   APT policy: (700, 'testing'), (650, 'unstable'), (600, 
'experimental'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 6.11.5-amd64 (SMP w/8 CPU threads; PREEMPT)
Locale: LANG=de_DE.utf8, LC_CTYPE=de_DE.utf8 (charmap=UTF-8), LANGUAGE 
not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages josm depends on:
ii  default-jre [java11-runtime]     2:1.21-76
ii  fonts-noto                       20201225-2
ii  jmapviewer                       2.22+dfsg-1
ii  libcommons-compress-java         1.27.1-2
ii  libgettext-commons-java          0.9.6-6
ii  openjdk-11-jre [java11-runtime]  11.0.25+9-1
ii  openjdk-14-jre [java11-runtime]  14.0.2+12-2
ii  openjdk-17-jre [java11-runtime]  17.0.13+11-2
ii  openjdk-21-jre [java11-runtime]  21.0.5+11-1
ii  proj-data                        9.5.0-1

Versions of packages josm recommends:
ii  josm-l10n  0.0.svn19253+dfsg-1
ii  openjfx    11.0.11+1-3.2

josm suggests no packages.

-- Configuration Files:
/etc/default/josm changed [not included]

-- no debconf information

Java version:

$ java -version
openjdk version "21.0.5" 2024-10-15
OpenJDK Runtime Environment (build 21.0.5+11-Debian-1)
OpenJDK 64-Bit Server VM (build 21.0.5+11-Debian-1, mixed mode, sharing)