Bug#677968: [DFS11732] - CPL - Segmentation faults in tests.

[Ole Streicher]

I investigated this a bit more for cpl_mast_test.

The cause of the error is a pointer misalignment in several cpl_mask
structure (and probaly elsewhere).

cpl_mask.c contains lines like (cpl_mask.c, line 513, for the 2_2 case):

#define CPL_MASK_FILTER_WORD \
    self[i] = \
[...]
        (OPERATE_WORD(*(const size_t*)(other + i + nx  )) & kernelw[2])
[...]

"other" (the data pointer in a mask) is a "const cpl_binary *", with
sizeof(cpl_binary)==1. "nx" is one dimension of the mask; an arbitrary
value which is in the test set to 21. "i" may be set to 2.

"other" is well-aligned to 2 bytes; "other + i + nx" is it therefore
*not*. Trying to dereference this as a "size_t" pointer fails on
machines that are sensitive to misaligned data, where Sparc is an example.

The problem is, however, not limited to Sparc processors. On other
machines (x86_64), it may lead to significant performance decrease.

This code will also create a rollover on the borders of the "other"
mask: each code line (as the one shown above) shall obviously be limited
to one mask line which has a length of nx == 21. CPL_MASK_FILTER_WORD,
however runs for i=2 to istop (cpl_mask_body.h:109), with istop set to
19. So, with sizeof(size_t)==4, for the last i the code already takes
the next mask line!

I would consider this code part as *buggy* even if it may run on
occasionally platforms. It also contradicts the C standard [(SO/IEC
9899:2011), section 6.3.2.3, para. 7:

    A pointer to an object or incomplete type may be converted to a
pointer to a different object or incomplete type. If the resulting
pointer is not correctly aligned for the pointed-to type, the behavior
is undefined.

Best regards

Ole