[Debian-science-sagemath] Overview of doctest failures

[Ximin Luo]

Tobias Hansen:
> Hi all,
> 
> I created a new status page to keep an eye on which doctests fail on
> which architectures:
> 
> https://people.debian.org/~thansen/sage-test-status.html
> 
> One use case is to identify which tests fail only when run in parallel.
> (There are a number of those and for sagemath 8.0-8 that lead to all
> tests being rerun in series on most architectures).
> 
> The page is updated every night together with the status page of package
> versions.
> 

Meant to say this earlier - this is really cool! Makes things a lot more convenient. Using it, I've noticed one weird thing with our test runs:

We run all the tests in parallel, then run again all failed tests (-f) in serial. However it looks like everywhere (in unstable and experimental, on all arches) src/sage/matrix/matrix_double_dense.pyx is not getting run the second time:

- https://people.debian.org/~thansen/sage-test-status-experimental.html
  arm64 has 9 failures parallel, 0 failures serial
- https://people.debian.org/~thansen/sage-test-status.html
  as above, but also for armhf i386 mips64el ppc64el

If you examine the logs manually:
- https://buildd.debian.org/status/fetch.php?pkg=sagemath&arch=arm64&ver=8.1%7Ebeta7-1&stamp=1508252992&raw=1
- https://buildd.debian.org/status/fetch.php?pkg=sagemath&arch=i386&ver=8.0-8&stamp=1505901595&raw=1

Indeed src/sage/matrix/matrix_double_dense.pyx does not show up in the "failures summary" at the end of the 2nd test run. So your page-generation script is working correctly. However if you grep upwards for the test name, it does not show the test being run in XXX seconds, like it does for the 1st test run. That suggests it's not getting run at all, for some reason.

However another test *is* getting run, that is src/sage/libs/gap/all_documented_functions.py

It has 1 failure in the 1st test run but 0 failures in the 2nd test run (on several suites and platforms, as with matrix_double_dense.pyx).

But if you grep upwards for the test name, you *do* find that the 2nd test run explicitly states that it was run in XXX seconds.

Another weird observation is that matrix_double_dense.pyx runs and fails in both test runs for mipsel, i.e. it is not omitted from the 2nd test run.

I understand there is more important work to do on the package besides investigate this, but I thought I'd report this detail so that people are aware the second test run may be missing some things.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git