Bug#742148: shapelib: FTBFS on powerpc (Both BIG_ENDIAN and LITTLE_ENDIAN defined!

[Francesco P. Lovergine]

On Wed, Mar 26, 2014 at 02:17:50PM +0100, Cyril Brulebois wrote:
> Francesco P. Lovergine <frankie at debian.org> (2014-03-26):
> > While I accepted the patch a few minutes ago, indeed I seriously now doubt that
> > the fix is correct.
> > 
> > It seems to me that in the original program the LITTLE_ENDIAN should be
> > intended as a static build-time definition for the host where the program is built.
> > 
> > So the NAN definition should be intended instead as nan(). That's because
> > the shapefile format is endianess-independent and shapelib reads/writes fields on the
> > basis of the host platform to respect the format. So the NAN should be
> > intended as the *host* NaN format, indeed (to be translated in the shp format NaN, i.e. little endian). 
> > That poses a problem on the pcc architecture for instance: __nan_bytes will be not the 
> > correct NaN value and will result as a big endian in the file.
> > 
> > See http://dl.maptools.org/dl/shapelib/shapefile.pdf for format.
> > 
> > Do you agree?
> 
> To be frank I didn't quite get why it was considered a good idea to
> hardcode setting -Dfoo in contrib/Makefile unconditionally instead of
> looking at the relevant arch-specific bits. So I assumed this was
> deliberate and that this setting was orthogonal to what's in system
> headers, that's why I proposed the patch you saw.
> 
> (From a quick look between last two upstream releases, this part didn't
> change; I guess this issue popped up due to updated system headers, but
> I didn't look into it to see what exactly triggered it.)
> 

I guess the contrib stuff is not so well maintained and probably not
too much coherent.

> I guess looking at __BYTE_ORDER would be a better way to actually check
> a system's endianness, #error-ing if it's neither __LITTLE_ENDIAN or
> __BIG_ENDIAN; I have no idea how much that is portable, but upstream
> should probably now a bit about msvc and advise whether that's a viable
> option.
> 

Well, I would avoid to upset the upstream code that much, a simple use of nan()
instead of NAN could propagate correctly. My only doubt is about the possible 
inclusion of special IEEE values within the final shapefile, a condition that
should not be admitted on the basis of the specs. But this is bread for
upstream's teeth.

-- 
Francesco P. Lovergine