Bug#944769: python3-h5py fails to import if offline due to apparent MPI failure

Drew Parsons dparsons at debian.org
Sun Mar 1 19:26:41 GMT 2020


Source: h5py
Followup-For: Bug #944769

Hi Thibaut,

your h5py suggestion of
  if MPI_ACTIVE:
    from h5py_mpi import *
  else:
    from h5py_serial import *

works up to a point.

But it's not a full and simple solution because of the way Python
handles submodules.

Some of the internal files, like h5t.pyx or _hl/files.py, want to
import functions, as in 
  from .h5py_warnings import H5pyDeprecationWarning

Conceivably client programs may also want to import classes this way.
  
But Python can only import using the direct module name.  Here we're
aliasing h5py_mpi or h5py_serial as h5py. So we get
ModuleNotFoundError, e.g. (testing the serial build)

$ cd .pybuild/cpython3_3.7_h5py-serial/build
$ PYTHONPATH=`pwd` python3
Python 3.7.6 (default, Jan 19 2020, 22:34:52) 
[GCC 9.2.1 20200117] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import h5py
<module 'h5py_serial' from '/home/build/h5py/.pybuild/cpython3_3.7_h5py-serial/build/h5py_serial/__init__.py'>
>>> h5py.h5py_warnings
<module 'h5py_serial.h5py_warnings' from '/home/build/h5py/.pybuild/cpython3_3.7_h5py-serial/build/h5py_serial/h5py_warnings.py'>
>>> h5py.h5py_warnings.H5pyDeprecationWarning
<class 'h5py_serial.h5py_warnings.H5pyDeprecationWarning'>
>>> from h5py.h5py_warnings import H5pyDeprecationWarning
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'h5py.h5py_warnings'
>>> import h5py_serial
>>> from h5py_serial.h5py_warnings import H5pyDeprecationWarning
>>> 

So the submodule h5py.h5py_warnings is accessible, and the class
H5pyDeprecationWarning inside it is accessible.

But that class cannot be imported independently on its own via the
h5py namespace (since the actual module is h5py_serial. It imports
cleanly via h5py_serial)

A client program could work around around it by using 
  H5pyDeprecationWarning = h5py.h5py_warnings.H5pyDeprecationWarning
instead of
  from h5py.h5py_warnings import H5pyDeprecationWarning

But the module is not supposed to have that limitation so almost
certainly some code will break.  And we shouldn't break code this way.

This is a known problem with Python submodules, e.g.
https://stackoverflow.com/questions/40823418/why-cant-i-import-from-a-module-alias
https://stackoverflow.com/questions/12229580/python-importing-a-sub-package-or-sub-module
https://stackoverflow.com/questions/24302754/python-submodule-imports-using-init-py

(there are also some weak private members that should probably be
accessible, namely _objects, _hl, also _conv, _errors, _proxy. These
could be added manually but nice to automate the import )

Can you think of a clean solution that allows members like
H5pyDeprecationWarning to get imported?



More information about the debian-science-maintainers mailing list