Bug#1010760: minetest: Floating point serialization error on x86

[Nils Dagsson Moskopp]

Package: minetest
Version: 5.3.0+repack-2.1+deb11u1
Severity: normal
Tags: patch upstream
X-Debbugs-Cc: nils+debian-reportbug at dieweltistgarnichtso.net

Dear Maintainer,

Minetest in versions lower than 5.5.0 is being miscompiled by GCC on 
x86. To verify this bug, run Minetest using “minetest --run-unittests” 
on x86 and look for the following three lines:

Test assertion failed: readF1000(is) == 53.534f
    at test_serialization.cpp:308
[FAIL] testStreamRead - 0ms

The reason for this is that the x87 FPU computes in 80bit precision by 
default, while IEEE-754 requires 64 bit precision.

There exist two ways to solve this. Upstream forces SSE2 for floating 
point calculations, by adding the following to CMakeLists.txt on x86:

--- start of patch ---
# use SSE for floating-point operations to avoid issues with improper fp-rounding and loss of precision
# when moving fp-data to incompatible or less-precise registers/storage locations
# see https://gcc.gnu.org/wiki/FloatingPointMath and https://gcc.gnu.org/wiki/x87note

add_compile_options(-mfpmath=sse -msse2)
--- end of patch ---

A non-SSE2 way is to achieve this is to use the compiler option “-mpc64”. 
Both achieve the same goal, calculating with 64 bit precision instead of 
doing calculations with 80 bit precision and then rounding the result to 
64 bit (which makes the unit test fail).

I therefore suggest to instead try the following to CMakeLists.txt on x86:

--- start of patch ---
# Limit x87 FPU to 64 bit precision to avoid floating point precision 
# errors. See both https://gcc.gnu.org/wiki/FloatingPointMath and 
# https://gcc.gnu.org/wiki/x87note for more details about this.

add_compile_options(-mpc64)
--- end of patch ---

If you use the latter, verify that this makes the testStreamRead not fail.

There is no need to upstream this change, as it has already been fixed 
in Minetest 5.5.0, but since it affects serialization, it might lead to 
some crashes or maybe even security bugs if Minetest 5.3.x and 5.4.x are 
being distributed in ways that floating point calculations are miscompiled.

For full context, see <https://github.com/minetest/minetest/issues/11810>.

-- System Information:
Debian Release: 11.3
  APT prefers stable
  APT policy: (900, 'stable'), (500, 'oldoldstable')
Architecture: i386 (i686)

Kernel: Linux 5.10.0-10-686 (SMP w/2 CPU threads)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages minetest depends on:
ii  libc6             2.31-13+deb11u3
ii  libcurl3-gnutls   7.74.0-1.3+deb11u1
ii  libfreetype6      2.10.4+dfsg-1
ii  libgcc-s1         10.2.1-6
ii  libgmp10          2:6.2.1+dfsg-1+deb11u1
ii  libirrlicht1.8    1.8.4+dfsg1-1.1
ii  libjsoncpp24      1.9.4-4
ii  libleveldb1d      1.22-3
ii  libluajit-5.1-2   2.1.0~beta3+dfsg-5.3
ii  libncursesw6      6.2+20201114-2
ii  libopenal1        1:1.19.1-2
ii  libpq5            13.5-0+deb11u1
ii  libspatialindex6  1.9.3-2
ii  libsqlite3-0      3.34.1-3
ii  libstdc++6        10.2.1-6
ii  libtinfo6         6.2+20201114-2
ii  libvorbisfile3    1.3.7-1
ii  libx11-6          2:1.7.2-1
ii  minetest-data     5.3.0+repack-2.1+deb11u1
ii  zlib1g            1:1.2.11.dfsg-2

minetest recommends no packages.

Versions of packages minetest suggests:
pn  minetest-mod-moreblocks  <none>
pn  minetest-mod-moreores    <none>
pn  minetest-mod-pipeworks   <none>
pn  minetest-server          <none>
pn  minetestmapper           <none>

-- no debconf information