[Python-modules-team] Bug#861249: FTBFS: math_test.Vector2TypeTest.test_cross fails

[Edmund Grimley Evans]

On arm64 and at least one other architecture, the error says:

-3.2862601528904633e-16 != 0

It looks as though the test is computing (1.2 * 3.4 - 3.4 * 1.2).

Now, the log to base 2 of (1.2 * 3.4) divided by 3.286e-16 is about
53.5. There are 52 bits in the mantissa of a 64-bit float, or 53
including the implicit "1". So I would guess that the error comes from
multiply and subtract being combined into a single operation, so that
the error in computing 1.2 * 3.4 is revealed.

If this is a real problem, perhaps "cross" could be implemented in a
way that prevents multiply-subtract from being used. If it is not a
real problem, then the test should be made less sensitive, perhaps by
requiring the absolute value of the result to be less than 1e-12 or
some other arbitrary epsilon.