[Debian-astro-maintainers] Bug#917602: python-fitsio 0.9.11+dfsg-4: FTBFS, alignment problem

[Aurelien Jarno]

On 2018-12-29 01:40, Steve McIntyre wrote:
> Source: python-fitsio
> Version: 0.9.11+dfsg-4
> Severity: important
> User: debian-arm at lists.debian.org
> Usertags: alignment
> 
> Hi!
> 
> I've been doing a full rebuild of the Debian archive, building all
> source packages targeting armel and armhf using arm64 hardware. We are
> planning in future to move all of our 32-bit armel/armhf builds to
> using arm64 machines, so this rebuild is to identify packages that
> might have problems with this configuration.
> 
> A feature of the arm64 kernel is that it does *not* support fixing up
> code with broken alignment, so code that might have built and run OK
> on our older armel/armhf build machines due to kernel fixups will now
> fail.
> 
> When building your package, I've found a bus error (aka alignment
> fault). The full log is online at
> 
>   https://www.einval.com/debian/arm/rebuild-logs/armel/FAIL/python-fitsio_0.9.11+dfsg-4_armel.log
> 
> for reference
> 
> I've done a quick bit of debugging to find the source of the
> bug. Here's a gdb stacktrace to demonstrate the problem. I'm not sure
> if the likely culprit is the test code here, or the underlying
> library.
> 
> (sid-armel)steve at maul:~/debian/build/python-fitsio/python-fitsio-0.9.11+dfsg$ gdb /usr/bin/python2.7 ./.pybuild/cpython2_2.7_fitsio/build/core
> ...
> warning: Could not load shared library symbols for fitsio/_fitsio_wrap.so.
> Do you need "set solib-search-path" or "set sysroot"?
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/arm-linux-gnueabi/libthread_db.so.1".
> Core was generated by `python2.7 -m unittest discover -v'.
> Program terminated with signal SIGBUS, Bus error.
> #0  ffi8fi8 (input=input at entry=0x7, ntodo=ntodo at entry=1, scale=<optimized out>, zero=0, output=output at entry=0xff861368, status=status at entry=0xff868554) at putcolj.c:1900
> 1900    putcolj.c: No such file or directory.
> (gdb) bt
> #0  ffi8fi8 (input=input at entry=0x7, ntodo=ntodo at entry=1, scale=<optimized out>, zero=0, output=output at entry=0xff861368, status=status at entry=0xff868554) at putcolj.c:1900
> #1  0xf5074ae4 in ffpcljj (fptr=0x1, colnum=-149821080, firstrow=<optimized out>, firstelem=<optimized out>, nelem=1, array=0x2303cef, status=0xff868554) at putcolj.c:1442
> #2  0xf5074ce4 in ffpcljj (fptr=<optimized out>, colnum=<optimized out>, firstrow=<optimized out>, firstelem=1, nelem=1, array=0x2303cef, array at entry=0x1, status=status at entry=0xff868554)
>     at putcolj.c:1371
> #3  0xf50669c4 in ffpcl (status=0xff868554, array=0x1, nelem=1, firstelem=1, firstrow=-768257499031892296, colnum=<optimized out>, datatype=<optimized out>, fptr=<optimized out>)
>     at putcol.c:739
> #4  ffpcl (fptr=<optimized out>, datatype=<optimized out>, colnum=<optimized out>, firstrow=1, firstelem=1, nelem=1, array=0x2303cef, status=0xff868554) at putcol.c:668
> #5  0xf55699e0 in ?? ()
> Backtrace stopped: previous frame identical to this frame (corrupt stack?)

The problem comes from the following code in PyFITSObject_write_columns():

|     for (irow=0; irow<nelem; irow++) {
|         thisrow = firstrow + irow;
|         for (icol=0; icol<ncols; icol++) {
|             data=PyArray_GETPTR1(array_ptrs[icol], irow);

Quoting the Numpy manual:

| You may need to take care when accessing the data in the array,
| however, if it is not in machine byte-order, misaligned, or not
| writeable. In other words, be sure to respect the state of the flags
| unless you know what you are doing, or have previously guaranteed an
| array that is writeable, aligned, and in machine byte-order using
| PyArray_FromAny.

So in addition to the alignment issue, it means that the function does
not work on non contiguous array or less likely, on non machine
byte-order arrays. The problem is therefore not limited to arm.

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien at aurel32.net                 http://www.aurel32.net