Bug#1146431: apache-arrow: builds only against the default python version
Maximiliano Curia
maxy at debian.org
Tue Sep 1 20:45:19 BST 2026
Package: src:apache-arrow
Version: 25.0.0-2
User: debian-python at lists.debian.org
Usertags: python3.15
Tags: patch, forky, sid
Hi!
While rebuilding the python related packages against the Python 3.15rc1
version we found that apache-arrow builds correctly, but only against
the default python version. Causing another package that could use more
than one python version to fail (dask).
To solve this, a number of changes needed to be applied:
- Patch calls to set_target_properties to use EXT_SUFFIX
- Update links in pyarrow-dev to point to the default version
- Build depend on python-all-dev
- Update test to run against the supported python versions
I applied these fixes in the sandbox [1] to be able to build the
packages that depend on apache-arrow, please consider applying the
patches to make it possible to build against multiple Python versions.
Happy hacking,
[1]: https://debusine.debian.net/debian/r-python-python3.15/
--
"Can you imagine what I would do if I could do all I can?" -- Sun Tzu
Saludos /\/\ /\ >< `/
-------------- next part --------------
diff -Nru apache-arrow-25.0.0/debian/changelog apache-arrow-25.0.0/debian/changelog
--- apache-arrow-25.0.0/debian/changelog 2026-08-02 12:48:53.000000000 +0200
+++ apache-arrow-25.0.0/debian/changelog 2026-09-01 13:17:56.000000000 +0200
@@ -1,3 +1,17 @@
+apache-arrow (25.0.0-2.1) UNRELEASED; urgency=medium
+
+ [ Maximiliano Curia ]
+ * Non-maintainer upload.
+ * Add patch to support multi version library suffix.
+ * Build against python3-all-dev
+ * Update install files and rules to handle multi version.
+ * Update test to run against all the python3 supported versions.
+
+ [ Dominique Belhachemi ]
+ * Mark gandiva-riscv64-pic-jit.patch as forwarded
+
+ -- Maximiliano Curia <maxy at debian.org> Tue, 01 Sep 2026 13:17:56 +0200
+
apache-arrow (25.0.0-2) unstable; urgency=medium
* add gandiva-riscv64-pic-jit.patch: build the riscv64 Gandiva JIT PIC,
@@ -52,7 +66,7 @@
apache-arrow (23.0.1-8) unstable; urgency=medium
- * source upload
+ * source upload
-- Rene Engelhard <rene at debian.org> Sat, 14 Mar 2026 23:05:41 +0100
diff -Nru apache-arrow-25.0.0/debian/control apache-arrow-25.0.0/debian/control
--- apache-arrow-25.0.0/debian/control 2026-07-24 19:17:20.000000000 +0200
+++ apache-arrow-25.0.0/debian/control 2026-09-01 13:17:56.000000000 +0200
@@ -43,7 +43,7 @@
ninja-build,
protobuf-compiler-grpc,
pybuild-plugin-pyproject,
- python3-dev,
+ python3-all-dev,
python3-numpy,
python3-scikit-build-core,
python3-setuptools-scm,
diff -Nru apache-arrow-25.0.0/debian/patches/gandiva-riscv64-pic-jit.patch apache-arrow-25.0.0/debian/patches/gandiva-riscv64-pic-jit.patch
--- apache-arrow-25.0.0/debian/patches/gandiva-riscv64-pic-jit.patch 2026-08-02 12:48:53.000000000 +0200
+++ apache-arrow-25.0.0/debian/patches/gandiva-riscv64-pic-jit.patch 2026-09-01 13:17:56.000000000 +0200
@@ -1,7 +1,7 @@
Description: [Gandiva] build the riscv64 JIT with the PIC relocation model
Fixes R_RISCV_HI20 "out of range" test_gandiva failures on riscv64.
Author: Dominique Belhachemi <domibel at debian.org>
-Forwarded: no
+Forwarded: https://github.com/apache/arrow/pull/50799
Last-Update: 2026-08-02
--- a/cpp/src/gandiva/engine.cc
+++ b/cpp/src/gandiva/engine.cc
diff -Nru apache-arrow-25.0.0/debian/patches/python-multi-version-library-suffix.patch apache-arrow-25.0.0/debian/patches/python-multi-version-library-suffix.patch
--- apache-arrow-25.0.0/debian/patches/python-multi-version-library-suffix.patch 1970-01-01 01:00:00.000000000 +0100
+++ apache-arrow-25.0.0/debian/patches/python-multi-version-library-suffix.patch 2026-09-01 13:17:56.000000000 +0200
@@ -0,0 +1,43 @@
+Description: Use Python extension suffix for PyArrow C++ libraries
+ When building PyArrow for multiple Python versions, the C++ helper libraries
+ (libarrow_python, libarrow_python_flight, libarrow_python_parquet_encryption)
+ must use Python's EXT_SUFFIX to prevent conflicts between different Python
+ interpreter versions.
+Author: Maximiliano Curia <maxy at debian.org>
+Forwarded: not-needed
+Last-Update: 2026-09-01
+
+Index: arrow/python/CMakeLists.txt
+===================================================================
+--- arrow.orig/python/CMakeLists.txt
++++ arrow/python/CMakeLists.txt
+@@ -530,8 +530,7 @@ target_compile_definitions(arrow_python
+ if(CMAKE_CONFIGURATION_TYPES)
+ target_compile_definitions(arrow_python PRIVATE PYARROW_BUILD_TYPE="$<CONFIG>")
+ endif()
+-set_target_properties(arrow_python PROPERTIES VERSION "${PYARROW_FULL_SO_VERSION}"
+- SOVERSION "${PYARROW_SO_VERSION}")
++set_target_properties(arrow_python PROPERTIES SUFFIX "${_EXT_SUFFIX}")
+ install(TARGETS arrow_python
+ ARCHIVE DESTINATION .
+ LIBRARY DESTINATION .
+@@ -548,8 +547,7 @@ else()
+ target_compile_definitions(arrow_python_parquet_encryption
+ PRIVATE ARROW_PYTHON_PARQUET_ENCRYPTION_EXPORTING)
+ set_target_properties(arrow_python_parquet_encryption
+- PROPERTIES VERSION "${PYARROW_FULL_SO_VERSION}"
+- SOVERSION "${PYARROW_SO_VERSION}")
++ PROPERTIES SUFFIX "${_EXT_SUFFIX}")
+ install(TARGETS arrow_python_parquet_encryption
+ ARCHIVE DESTINATION .
+ LIBRARY DESTINATION .
+@@ -581,8 +579,7 @@ if(PYARROW_BUILD_FLIGHT)
+ ArrowFlight::arrow_flight_shared)
+ target_compile_definitions(arrow_python_flight PRIVATE ARROW_PYFLIGHT_EXPORTING)
+ set_target_properties(arrow_python_flight
+- PROPERTIES VERSION "${PYARROW_FULL_SO_VERSION}"
+- SOVERSION "${PYARROW_SO_VERSION}")
++ PROPERTIES SUFFIX "${_EXT_SUFFIX}")
+ install(TARGETS arrow_python_flight
+ ARCHIVE DESTINATION .
+ LIBRARY DESTINATION .
diff -Nru apache-arrow-25.0.0/debian/patches/series apache-arrow-25.0.0/debian/patches/series
--- apache-arrow-25.0.0/debian/patches/series 2026-08-02 12:48:41.000000000 +0200
+++ apache-arrow-25.0.0/debian/patches/series 2026-09-01 13:17:56.000000000 +0200
@@ -1,3 +1,4 @@
use-debian-mimalloc.patch
build-sse4.2-kernels.patch
gandiva-riscv64-pic-jit.patch
+python-multi-version-library-suffix.patch
diff -Nru apache-arrow-25.0.0/debian/pyarrow-dev.install apache-arrow-25.0.0/debian/pyarrow-dev.install
--- apache-arrow-25.0.0/debian/pyarrow-dev.install 2026-07-18 18:48:35.000000000 +0200
+++ apache-arrow-25.0.0/debian/pyarrow-dev.install 2026-09-01 13:17:56.000000000 +0200
@@ -1,2 +1,3 @@
usr/lib/python3/dist-packages/pyarrow/include
usr/lib/python3/dist-packages/pyarrow/src
+usr/lib/python3/dist-packages/pyarrow/libarrow_python*.so
diff -Nru apache-arrow-25.0.0/debian/pyarrow-dev.links apache-arrow-25.0.0/debian/pyarrow-dev.links
--- apache-arrow-25.0.0/debian/pyarrow-dev.links 2026-07-24 19:17:20.000000000 +0200
+++ apache-arrow-25.0.0/debian/pyarrow-dev.links 1970-01-01 01:00:00.000000000 +0100
@@ -1,3 +0,0 @@
-usr/lib/python3/dist-packages/pyarrow/libarrow_python.so.2500 usr/lib/python3/dist-packages/pyarrow/libarrow_python.so
-usr/lib/python3/dist-packages/pyarrow/libarrow_python_flight.so.2500 usr/lib/python3/dist-packages/pyarrow/libarrow_python_flight.so
-usr/lib/python3/dist-packages/pyarrow/libarrow_python_parquet_encryption.so.2500 usr/lib/python3/dist-packages/pyarrow/libarrow_python_parquet_encryption.so
diff -Nru apache-arrow-25.0.0/debian/python3-pyarrow.install apache-arrow-25.0.0/debian/python3-pyarrow.install
--- apache-arrow-25.0.0/debian/python3-pyarrow.install 2026-07-24 19:17:20.000000000 +0200
+++ apache-arrow-25.0.0/debian/python3-pyarrow.install 2026-09-01 13:17:56.000000000 +0200
@@ -1,3 +1,2 @@
usr/lib/python3.*/dist-packages/pyarrow-*.dist-info/*
usr/lib/python3.*/dist-packages/pyarrow/*
-usr/lib/python3/dist-packages/pyarrow/libarrow_python*.so.*
diff -Nru apache-arrow-25.0.0/debian/python3-pyarrow.links apache-arrow-25.0.0/debian/python3-pyarrow.links
--- apache-arrow-25.0.0/debian/python3-pyarrow.links 2026-07-24 19:17:20.000000000 +0200
+++ apache-arrow-25.0.0/debian/python3-pyarrow.links 1970-01-01 01:00:00.000000000 +0100
@@ -1,3 +0,0 @@
-usr/lib/python3/dist-packages/pyarrow/libarrow_python.so.2500.0.0 usr/lib/python3/dist-packages/pyarrow/libarrow_python.so.2500
-usr/lib/python3/dist-packages/pyarrow/libarrow_python_flight.so.2500.0.0 usr/lib/python3/dist-packages/pyarrow/libarrow_python_flight.so.2500
-usr/lib/python3/dist-packages/pyarrow/libarrow_python_parquet_encryption.so.2500.0.0 usr/lib/python3/dist-packages/pyarrow/libarrow_python_parquet_encryption.so.2500
diff -Nru apache-arrow-25.0.0/debian/rules apache-arrow-25.0.0/debian/rules
--- apache-arrow-25.0.0/debian/rules 2026-07-24 19:17:20.000000000 +0200
+++ apache-arrow-25.0.0/debian/rules 2026-09-01 13:17:56.000000000 +0200
@@ -19,7 +19,7 @@
SOVERSION=2500
PY3DEFAULT := $(shell py3versions -d)
-export PYBUILD_VERSIONS = $(shell py3versions -dv)
+PY3DEFAULT_EXT_SUFFIX := $(shell $(PY3DEFAULT) -c "import sysconfig; print(sysconfig.get_config_var('EXT_SUFFIX'))")
export PYBUILD_SYSTEM = pyproject
NUMJOBS := $(or $(DEB_BUILD_OPTION_PARALLEL),1)
@@ -46,7 +46,7 @@
# Without build-dir, scikit-build-core builds in a fresh mkdtemp directory whose
# name reaches the extensions' debug info and python3-pyarrow stops being reproducible.
-export PYBUILD_BUILD_ARGS = --config-setting build-dir=$(CURDIR)/python_build $(PYARROW_CMAKE_ARGS)
+export PYBUILD_BUILD_ARGS = --config-setting build-dir=$(CURDIR)/python_build_{version} $(PYARROW_CMAKE_ARGS)
BUILD_TYPE=release
@@ -185,17 +185,20 @@
rm -rf debian/tmp/usr/lib/python*/dist-packages/.pytest_cache/
# Move non-Python-version-specific files from python3.X/ to python3/
mkdir -p debian/tmp/usr/lib/python3/dist-packages/pyarrow/
- # Bridge libs: move from PY3DEFAULT, keep out of python3.X/ so dh_python3 ignores them
- mv debian/tmp/usr/lib/$(PY3DEFAULT)/dist-packages/pyarrow/libarrow_python*.so.* \
- debian/tmp/usr/lib/python3/dist-packages/pyarrow/
- # drop the 2 duplicate bridge-lib copies; python3-pyarrow.links recreates the .so.$(SOVERSION) symlink
- rm -f debian/tmp/usr/lib/$(PY3DEFAULT)/dist-packages/pyarrow/libarrow_python*.so
- rm -f debian/tmp/usr/lib/python3/dist-packages/pyarrow/libarrow_python*.so.$(SOVERSION)
# include/ and src/: move from PY3DEFAULT
mv debian/tmp/usr/lib/$(PY3DEFAULT)/dist-packages/pyarrow/include \
debian/tmp/usr/lib/python3/dist-packages/pyarrow/
mv debian/tmp/usr/lib/$(PY3DEFAULT)/dist-packages/pyarrow/src \
debian/tmp/usr/lib/python3/dist-packages/pyarrow/
+ rm -rf debian/tmp/usr/lib/python3.*/dist-packages/pyarrow/include
+ rm -rf debian/tmp/usr/lib/python3.*/dist-packages/pyarrow/src
+ # Dev symlinks for default Python version (for pyarrow-dev)
+ ln -sf libarrow_python$(PY3DEFAULT_EXT_SUFFIX) \
+ debian/tmp/usr/lib/python3/dist-packages/pyarrow/libarrow_python.so
+ ln -sf libarrow_python_flight$(PY3DEFAULT_EXT_SUFFIX) \
+ debian/tmp/usr/lib/python3/dist-packages/pyarrow/libarrow_python_flight.so
+ ln -sf libarrow_python_parquet_encryption$(PY3DEFAULT_EXT_SUFFIX) \
+ debian/tmp/usr/lib/python3/dist-packages/pyarrow/libarrow_python_parquet_encryption.so
override_dh_auto_test:
@@ -227,7 +230,7 @@
rm -rf .pybuild .pytest_cache
rm -rf cpp_build
- rm -rf python_build
+ rm -rf python_build*
rm -f r/src/Makevars
rm -rf r_arrow_build
find . -name "*bin" -type f -delete
@@ -258,7 +261,6 @@
dh_makeshlibs -plibarrow-flight-sql$(SOVERSION)
dh_makeshlibs -plibparquet$(SOVERSION)
dh_makeshlibs -plibgandiva$(SOVERSION)
- dh_makeshlibs -ppython3-pyarrow
override_dh_shlibdeps:
dh_shlibdeps -Npython3-pyarrow \
diff -Nru apache-arrow-25.0.0/debian/tests/control apache-arrow-25.0.0/debian/tests/control
--- apache-arrow-25.0.0/debian/tests/control 2026-07-24 19:17:20.000000000 +0200
+++ apache-arrow-25.0.0/debian/tests/control 2026-09-01 13:17:56.000000000 +0200
@@ -11,7 +11,7 @@
python3-fsspec,
cython3 (>= 3.1),
g++,
- python3-dev,
+ python3-all-dev,
python3-setuptools,
python3-pandas,
python3-pytz,
diff -Nru apache-arrow-25.0.0/debian/tests/test-pyarrow apache-arrow-25.0.0/debian/tests/test-pyarrow
--- apache-arrow-25.0.0/debian/tests/test-pyarrow 2026-07-24 19:17:20.000000000 +0200
+++ apache-arrow-25.0.0/debian/tests/test-pyarrow 2026-09-01 13:17:56.000000000 +0200
@@ -12,7 +12,7 @@
EXTRA_ARGS="--forked"
fi
-for py in $(py3versions -d)
+for py in $(py3versions -s)
do
cd "$AUTOPKGTEST_TMP"
echo "Testing with $py:"
More information about the debian-science-maintainers
mailing list