[Debichem-devel] Bug#1027025: psi4: FTBFS with Python 3.11 as default version

[James Addison]

Source: psi4
Version: 1:1.3.2+dfsg-3
Followup-For: Bug #1027025
X-Debbugs-Cc: jay at jp-hosting.net

Based on some upstream discussion[1] of the same ImportError, it looks like this is
related to the PYTHON_MODULE_EXTENSION suffix applied to the 'core' module at build-time
by pybind11-related build infrastructure.

During these build failures, the module is compiled with filename 'coreNone' instead of
the expected filename 'core.<extension>', and that results in import-time errors.

The cause relates to a Python distutils.sysconfig config var named "SO" (used to
retrieve a compiled module's filename, including suffix), which was deprecated[2] in
Python 3.4 in favour of a var named "EXT_SUFFIX" that provides similar information.

The reason that psi4 v1.3.2 breaks in combination with Python 3.11 is that it uses a
variant of pybind11's 'FindPythonLibsNew' helper module that predates the
SO-to-EXT_SUFFIX transition (it references[3] only the former var) when determining
the value for PYTHON_MODULE_EXTENSION.  In Python 3.11, the SO var is no longer
available.

pybind11 handled this by providing a fallback[4] behaviour from v2.6.0, with some
further adjustments[5] applied in v2.6.1.

Upstream psi4 separately resolved the problem by commenting-out[6] a reference to
PYTHON_MODULE_EXTENSION from psi4 version v1.4 and subsequent versions.

[1] - https://github.com/psi4/psi4/issues/2656

[2] - https://docs.python.org/3/whatsnew/3.4.html?highlight=ext_suffix#deprecations-in-the-python-api

[3] - https://github.com/psi4/psi4/blob/ecbda834b4bcce794ce2338873962f36436842a5/cmake/FindPythonLibsNew.cmake#L81

[4] - https://github.com/pybind/pybind11/commit/ad2fe0ee7b5c8db7fe134ac568a507cc5bdb0617#diff-db8b532c178000412cf20931528e1e68b1d44cdfb0cbd4ca9abfc6e35f368994R118

[5] - https://github.com/pybind/pybind11/commit/3e4d54bc0181eb7aad2ac52a2a18be18c1ab9aee

[6] - https://github.com/psi4/psi4/pull/2048/commits/4074de6e76c933308ff092fd48a2420d84384b31#diff-d514c3ebd329db7a94a92197c524ff69f55581d6eff60f7013690e8d7483a353R150