Bug#986527: sagemath: FTBFS: how to address for Bullseye

[Adrian Bunk]

On Wed, Jun 09, 2021 at 12:32:02AM +0000, John Scott wrote:
>...
> With respect to this particular issue, I'd like to share my perspective
> wrangling with a package that poses a similar dilemma: GCC (I'm working
> on packaging gcc-sh-elf). Like the status quo with SageMath in Debian,
> GCC has a test suite where failures are normal, and in general it takes
> an individual to watch out for what number of failures counts as "too
> many." Rather than hardcode an arbitrary threshold for what number of
> failing tests is acceptable, it seems that it's much better, and in the
> interest of Debian ports and alternative build environments, to just
> let the tests run for informative purposes.

Note that sagemath already seems to have 2 different classes of failures:
https://buildd.debian.org/status/package.php?p=sagemath

On armhf, mips64el and ppc64el the build fails reliably with
  Error: critical test failures (e.g. timeout, segfault, etc.)
This seems to be a reasonable build-time smoke test.

The only special case is s390x:
  Checking number of failed tests to determine whether to rerun tests in series...
  No: 504 tests failed, up to 400 failures are tolerated for rerun
It is not a surprise that more tests a failing on big endian,
but there are actually no "critical test failures".

>...
> I believe it's in the best interest of Debian users that this bug be
> downgraded for Bullseye so Sage can be used in the mostly-wholesome
> shape it's in, but since I lack expertise in maintaining it I too will
> leave this to someone else.

Flaky builds are a pain, also for bullseye.

IMHO the best action would be an upload with the following changes:
- your superficial autopkgtest
- raising the limit from 200 to something that makes builds non-flaky
- optionally force build failure on s390x

I could sponsor or NMU such an upload.

cu
Adrian