Bug#953116: Bug#961108: openmpi: providing 64-bit MPI

[Drew Parsons]

Hi Gard, bringing this question over to petsc bug#953116.


On 2020-05-20 17:07, Gard Spreemann wrote:
> Drew Parsons <dparsons at debian.org> writes:
> 
>> If I understand correctly, it's dangerous to simply enable 64-bit in
>> PETSc alone. It needs to be done all along the computational library
>> stack.
> 
> In the case of PETSc, is the intention to change to using the 64-bit
> indexing option, or to provide a new additional package that uses 
> 64-bit
> indexing? The latter sounds burdensome long-term, so I would probably
> lean towards the former.

I was assuming we'd carry the two bit builds, "petsc-dev" and 
"petsc64-dev", at least in the medium term.  This would follow what's in 
place with BLAS.  It's also the practice in CRAY which offers both 
cray-petsc and cray-petsc-64 modules.

But it's a good question to consider.


> If you do intend to go for the former, I think it's a good idea to very
> clearly warn the user upon upgrade. The PETSc binary sparse matrix and
> vector file formats are assumed to use whatever indexing data types 
> that
> the library is compiled with, and as far as I remember there is no
> metadata about this in the file format itself. Without warning, I
> suspect a lot of users might burn themselves when using LoadMat [1] and
> friends on data files written with 32 bit index versions of the
> package. I don't know how common the file format is and whether most
> people just use HDF5, but I know I've often used the PETSc format to
> avoid the complications of HDF5.

Certainly.  I can anticipate it might be quite disruptive if the 
standard package just jumps to 64 bit. I imagine that would break 
things.


One question to consider is why petsc doesn't just use 64 bits in the 
first place on 64 bit systems.

I was under the impression that a 32 bit build actually runs faster on a 
64 bit system, in the sense of getting twice as much done per clock 
cycle. That you only need the 64 bit build if you actually need that 
much address space (i.e. if your mesh carrys that many degree of freedom 
DOFs)

I guess we should clear up whether that's true or not. It would be 
regrettable to drop 32 bit if it means performance on smaller jobs is 
diminished.

> PS: I can volunteer to write a 32->64 bit conversion tool for these
> files if desired. Ideally I guess upstream should provide 32/64-bit
> specific versions of the reading/writing functions.

That could be a helpful tool.  We could include it the -dev packages.

Drew