[Python-modules-team] Bug#982553: python3-aalib: ABI mismatch on amd64

[Celelibi]

Package: python3-aalib
Version: 0.3.2-4
Severity: normal

Dear Maintainer,

It looks like there's an ABI mismatch between the structure layout
declared in the python code and the actual layout.

It causes python to allocate less memory than needed by the C library.
On rare occasions, the uninitialized field gets a non-zero value, which
produce weird behaviors.

Specifically, the struct fields of aa_hardware_params and
aa_render_params are only aligned on 4 bytes on i386 when compiled with
gcc. Not on x64 or any other architecture as far as I can tell.

When reading the beginning of aalib.h we can see:

/* The -malign-double switch changes binarry compatibility with structures
   containing floating point values.  To avoid this, set alignment manually
   to old value.  */

#ifdef __GNUC__
#ifdef __i386__
#define __AA_ALIGN __attribute__ ((__aligned__ (4)))
#define __AA_NOALIGN __attribute__ ((__packed__))
#endif
#endif


Which shows explicitely that the struct fields are only aligned or
packed on i386. Given the specificity of the condition and the comment
above, my guess would be that this was made only to prevent an ABI
breakage when upgrading gcc, and not a general definition.

Therefore I think that the _pack_ = 4 attribute when declaring the
structures is too strict.

class HardwareSettings(Structure):
    _pack_ = 4
    _fields_ = [
        # ...
    ]


Although I have no clue how to detect it, this should probably be set
only for the 32 bits builds on intel-compatible platforms (only when
__i386__ is set). Note that 32 bits systems can be installed on 64 bits
hardware, and 32 bits builds of python can be installed in 64 bits
debian systems. So it should really be the ELFCLASS of the library that
should be tested, neither the architecture of the hardware nor the
debian arch.


On the other hand, maybe an easier fix would be to have a more
consistent ABI on the aalib side? Although this would mean bumping the
ABI version and possibly breaking many packaged, this might be the best
long-term solution. If you think so, feel free to reassign this bug
report as needed.



Best regards,
Celelibi


-- System Information:
Debian Release: bullseye/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'testing-debug'), (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 5.9.0-5-amd64 (SMP w/2 CPU threads)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages python3-aalib depends on:
ii  libaa1   1.4p5-48
ii  python3  3.9.1-1

python3-aalib recommends no packages.

Versions of packages python3-aalib suggests:
ii  python3-pil  8.1.0-1

-- no debconf information