Bug#944769: python3-h5py fails to import if offline due to apparent MPI failure

[Thibaut Paumard]

Hi Drew,

Le 27/01/2020 à 05:40, Drew Parsons a écrit :
>   from os import getenv as os_getenv
>   OPENMPI_MULTIPROC = os_getenv('OMPI_COMM_WORLD_SIZE') and int(os_getenv('OMPI_COMM_WORLD_SIZE')) > 1
>   MPICH_MULTIPROC = os_getenv('MPI_LOCALNRANKS') and int(os_getenv('MPI_LOCALNRANKS'))>1
>   MPI_ACTIVE = MPI and ( OPENMPI_MULTIPROC or MPICH_MULTIPROC )
>   
>   IF MPI_ACTIVE:
>       from mpi4py.MPI cimport MPI_Comm, MPI_Info, Comm, Info
> 
> 
> 
> But then during configuration cython fails:
> 
>   Error compiling Cython file:
[...]
>   h5py/api_types_ext.pxd:22:3: Compile-time name 'MPI_ACTIVE' not defined
> 

I have no experience with cython either but it looks like this capital
IF is compile-time, a bit like automake conditionals.

Maybe something like:


IF MPI_ACTIVE:
  from os import getenv as os_getenv
  OPENMPI_MULTIPROC = (os_getenv('OMPI_COMM_WORLD_SIZE') and
                       int(os_getenv('OMPI_COMM_WORLD_SIZE')) > 1)
  MPICH_MULTIPROC = (os_getenv('MPI_LOCALNRANKS') and
                     int(os_getenv('MPI_LOCALNRANKS'))>1)
  MPI_REALLY_ACTIVE = MPI and ( OPENMPI_MULTIPROC or MPICH_MULTIPROC )

  if MPI_REALLY_ACTIVE:
    from mpi4py.MPI cimport MPI_Comm, MPI_Info, Comm, Info

would work (i.e. include the full run-time test inside the compile-time
test). But of course the MPI stuff may be too much interleaved with the
rest of the code to be deactivated at run-time, in which case the best
option might be to provide two separate packages with and without MPI.

If it is possible to provide the two packages, you may even be able to
make them co-installable and add a very thin wrapper layer to do the
run-time check:

h5py.py:

if <running under mpirun>:
  from h5py_mpi import *
else:
  from h5py_nompi import *

Good luck with you exploration.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/debian-science-maintainers/attachments/20200129/90ddd016/attachment.sig>