Bug#979314: libccp4: The endianness check in ccp4_sysdep.h is incorrect assuming powerpc is always big endian

[Maarten L. Hekkelman]

Source: libccp4
Version: 6.5.1-4
Severity: grave
Tags: patch upstream
Justification: renders package unusable
X-Debbugs-Cc: maarten at hekkelman.com




-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: ppc64el (ppc64le)

Kernel: Linux 5.4.0-58-generic (SMP w/12 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: unable to detect


The file ccp4_sysdep.h checks for the endianness in an incorrect way,
the check assumes powerpc is always big endian. By moving the catch-all
test up the test is more robust.

regards, -maarten
-------------- next part --------------
--- a/ccp4/ccp4_sysdep.h
+++ b/ccp4/ccp4_sysdep.h
@@ -177,6 +177,26 @@
 #define DFNTF_CONVEXNATIVE 5    /**< Convex native floats */
 #define DFNTF_LEIEEE    4       /**< little-endian IEEE format */

+/* From time to time new architectures are added here, often because Linux
+ * packagers want to build it on all platforms supported by their distro.
+ * Here we try to catch machines not listed explicitely above, under
+ * assumption that endianness is the same for floating point numbers
+ * as for integers. Which is safe assumption on modern standard computers
+ * (not embedded systems), according to
+ * http://en.wikipedia.org/wiki/Endianness#Floating-point_and_endianness
+ */
+#if defined(__BYTE_ORDER)
+# if __BYTE_ORDER == __LITTLE_ENDIAN
+#  define NATIVEIT DFNTI_IBO
+#  define NATIVEFT DFNTF_LEIEEE
+# elif __BYTE_ORDER == __BIG_ENDIAN
+#  define NATIVEIT DFNTI_MBO
+#  define NATIVEFT DFNTF_BEIEEE
+# endif
+#endif
+
+#if !defined(NATIVEIT) && !defined(NATIVEFT)
+
 #if defined (VAX) || defined (vax) /* gcc seems to use vax */
 #  define NATIVEFT DFNTF_VAX
 #  define NATIVEIT DFNTI_IBO
@@ -222,22 +242,6 @@
 # endif
 #endif

-/* From time to time new architectures are added here, often because Linux
- * packagers want to build it on all platforms supported by their distro.
- * Here we try to catch machines not listed explicitely above, under
- * assumption that endianness is the same for floating point numbers
- * as for integers. Which is safe assumption on modern standard computers
- * (not embedded systems), according to
- * http://en.wikipedia.org/wiki/Endianness#Floating-point_and_endianness
- */
-#if !defined(NATIVEIT) && !defined(NATIVEFT) && defined(__BYTE_ORDER)
-# if __BYTE_ORDER == __LITTLE_ENDIAN
-#  define NATIVEIT DFNTI_IBO
-#  define NATIVEFT DFNTF_LEIEEE
-# elif __BYTE_ORDER == __BIG_ENDIAN
-#  define NATIVEIT DFNTI_MBO
-#  define NATIVEFT DFNTF_BEIEEE
-# endif
 #endif

 #ifndef NATIVEFT