[Pkg-rust-maintainers] Bug#1078430: rust-just: flaky autopkgtest on s390x: *nearly* always fails

[Blair Noctis]

On 10/08/2024 18:04, Paul Gevers wrote:
> Source: rust-just
> Version: 1.33.0-1
> Severity: serious
> User: debian-ci at lists.debian.org
> Usertags: flaky
> 
> Dear maintainer(s),
> 
> I looked at the results of the autopkgtest of your package because it was
> blocking the migration of gcc-14. I noticed that it nearly always fails on
> s390x, but not always. It's a bit flaky on other architectures as well.
> 
> Because the unstable-to-testing migration software now blocks on
> regressions in testing, flaky tests, i.e. tests that flip between
> passing and failing without changes to the list of installed packages,
> are causing people unrelated to your package to spend time on these
> tests.
> 
> Don't hesitate to reach out if you need help and some more information
> from our infrastructure.
> 
> Paul
> 
> https://ci.debian.net/packages/r/rust-just/testing/s390x/50327770/
> 
> 149s failures:
> 149s
> 149s ---- choose::status_error stdout ----
> 149s thread 'choose::status_error' panicked at tests/choose.rs:209:3:
> 149s assertion failed: Regex::new("^error: Chooser `exit-2` failed: exit (code|
> status): 2\n$").unwrap().is_match(str::from_utf8(&output.stderr).unwrap())
(...)

Hi Paul,

Thanks for bringing attention to it. I ran the tests a few times on zelenka and
on debci, which all passed. Since it's kind of a test-Heisenbug with no report
in real usage, I'll mark the tests ignored as a temporary workaround.

On the infra side, I'm pondering the idea of a "debug" patch that captures more
information the next time it goes south. However, "fixing" (or rather frankly,
ignoring) the test failures makes following runs pass. Is there a way to monitor
such an output in passed runs, other than downloading every run and grepping
through them?

-- 
Sdrager,
Blair Noctis
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/pkg-rust-maintainers/attachments/20240811/73d28d8a/attachment-0001.sig>