Bug#1106380: libtest2-harness-perl: FTBFS randomly: flaky tests
Santiago Vila
sanvila at debian.org
Fri May 23 22:28:59 BST 2025
Package: src:libtest2-harness-perl
Version: 1.000158-1
Severity: important
Tags: ftbfs trixie sid
Dear maintainer:
During a rebuild of all packages in unstable, your package failed to build:
[ please read notes at the end ]
--------------------------------------------------------------------------------
[...]
debian/rules clean
dh clean
dh_clean
debian/rules binary
dh binary
dh_update_autotools_config
dh_autoreconf
dh_auto_configure
/usr/bin/perl Makefile.PL INSTALLDIRS=vendor "OPTIMIZE=-g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -Wdate-time -D_FORTIFY_SOURCE=2" "LD=x86_64-linux-gnu-gcc -g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -Wl,-z,relro -Wl,-z,now"
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for Test2::Harness
Writing MYMETA.yml and MYMETA.json
[... snipped ...]
ok 3 - State matches expectations
1..3
}
ok 5 - consistent state {
ok 1 - state [1] matches state [0]
ok 2 - state [2] matches state [0]
ok 3 - State matches expectations
1..3
}
ok 6 - Cannot write after timing out
1..6
}
ok 6 - _entry_expired {
ok 1 - Invalid entry is expired
ok 2 - Entry to be removed is expired
ok 3 - no 'seen' field expired
ok 4 - Recently seen, not expired
ok 5 - Old is expired
1..5
}
ok 7 - runner_todo {
ok 1 - Nothing to do
ok 2 - Nothing to do
ok 3 - Got job count
ok 4 - todo is set
ok 5 - Got job count
ok 6 - todo is set
ok 7 - Got job count
ok 8 - todo is set
ok 9 - Got job count
ok 10 - todo is set
ok 11 - Got job count
ok 12 - todo is set
1..12
}
ok 8 - _runner_calcs {
ok 1 - Get cache if it is present
ok 2 - Calculated data
ok 3 - Have a cache
ok 4 - Result matches cache
ok 5 - Result matches cache
ok 6 - Extra cache key found
ok 7 - Calculated data
ok 8 - Calculated data
1..8
}
ok 9 - allocate_slots {
ok 1 - con must be specified
ok 2 - Cannot exceed slot limits A
ok 3 - Cannot exceed slot limits B
ok 4 - Cannot exceed slot limits C
ok 5 - runner calc cache is as expected
ok 6 - Allocation is 5
ok 7 - We got 4 slots!
ok 8 - Allocating slots reset runner calc cache
ok 9 - Allocation updated to 4
1..9
}
1..9
ok
t/unit/Test2/Harness/Settings.t ...............................
# Seeded srand with seed '20250523' from local date.
ok 1 - Created an instance
ok 2 - foo is not defined
ok 3 - Cannot call foo if it is not defined
ok 4 - Cannot call prefix(foo) if it is not defined
ok 5 - Defined the prefix
ok 6 - foo is now defined
ok 7 - Can call foo if it is defined
ok 8 - Can call prefix(foo) if it is defined
ok 9 - TO_JSON
ok 10 - Need a blessed instance
ok 11 - No args
ok 12 - Got a blessed instance of XXX
ok 13 - Instance is composed as expected
ok 14 - Correct class
ok 15 - Serialized and deserialized round trip
ok 16 - 2 different refs
ok 17 - Prefixes must be hashes
ok 18 - Blessed Prefixes must be prefixes
1..18
ok
t/unit/Test2/Harness/Settings/Prefix.t ........................
# Seeded srand with seed '20250523' from local date.
ok 1 - Created an instance
ok 2 - Hash is slightly obscured by an extra deref
ok 3 - Must use a valid field
ok 4 - vivify returns a ref
ok 5 - Not set yet
ok 6 - Set value
ok 7 - Set via lvalue
ok 8 - Got via field
ok 9 - Set via field
ok 10 - Field only takes 2 args
ok 11 - Autload does not work on class
ok 12 - JSON structure
ok 13 - Created an instance
ok 14 - Created with args
1..14
ok
t/unit/Test2/Harness/TestFile.t ...............................
# Seeded srand with seed '20250523' from local date.
ok 1 - use Test2::Harness::TestFile;
ok 2 - timeouts {
ok 1 - set event timeout
ok 2 - set event timeout
ok 3 - event timeout made it to task
ok 4 - post-exit timeout made it to task
ok 5 - set event timeout
ok 6 - set event timeout
ok 7 - Got warnings
1..7
}
ok 3 - invalid {
ok 1 - Need a valid test file
1..1
}
ok 4 - meta {
ok 1 - No key returns empty list
ok 2 - Empty key returns empty list
ok 3 - Got both values for the 'mykey' key
ok 4 - Got other key
ok 5 - Got hyphenated key
1..5
}
ok 5 - foo {
ok 1 - Category is foo
ok 2 - Stage is FoO, case-sensitive
1..2
}
ok 6 - package {
ok 1 - No preload
1..1
}
ok 7 - taint {
ok 1 - No SHBANG switches
ok 2 - Parsed shbang
ok 3 - Got queue item data
1..3
}
ok 8 - warn {
ok 1 - got SHBANG switches
ok 2 - Parsed shbang
ok 3 - Got queue item data
1..3
}
ok 9 - notime {
ok 1 - Timeouts turned off
ok 2 - Timeouts turned off with default 1
ok 3 - Category is general
ok 4 - Duration is long
ok 5 - No SHBANG switches
ok 6 - No shbang
ok 7 - Got queue item data
1..7
}
ok 10 - all {
ok 1 - Timeouts turned off
ok 2 - Timeouts turned off with default 1
ok 3 - Forking is off
ok 4 - Checking fork with different default
ok 5 - Preload is off
ok 6 - Checking preload with different default
ok 7 - No isolation
ok 8 - Use isolation with a default of false
ok 9 - Use stream
ok 10 - no stream with a default of true
ok 11 - Category is isolation
ok 12 - No SHBANG switches
ok 13 - No shbang
ok 14 - Got queue item data
1..14
}
ok 11 - med2 {
ok 1 - Timeouts turned on
ok 2 - Timeouts turned off with default 0
ok 3 - Forking is off
ok 4 - Checking fork with different default
ok 5 - Preload is on
ok 6 - Checking preload with different default
ok 7 - No isolation
ok 8 - Use isolation with a default of true
ok 9 - Use stream
ok 10 - no stream with a default of false
ok 11 - Category is general
ok 12 - duration is medium
ok 13 - No SHBANG switches
ok 14 - No shbang
ok 15 - Got queue item data
1..15
}
ok 12 - med1 {
ok 1 - Timeouts turned on
ok 2 - Timeouts turned off with default 0
ok 3 - Forking is ok
ok 4 - Checking fork with different default
ok 5 - Preload is off
ok 6 - Checking preload with different default
ok 7 - No isolation
ok 8 - Use isolation with a default of true
ok 9 - Use stream
ok 10 - no stream with a default of false
ok 11 - Category is general
ok 12 - duration is medium
ok 13 - No SHBANG switches
ok 14 - No shbang
ok 15 - Got queue item data
1..15
}
ok 13 - long {
ok 1 - Timeouts turned off
ok 2 - Timeouts turned off even with default 1
ok 3 - Forking is ok
ok 4 - Checking fork with different default
ok 5 - Preload is ok
ok 6 - Checking preload with different default
ok 7 - Use isolation
ok 8 - Use isolation even with a default of false
ok 9 - Use stream
ok 10 - no stream with a default of false
ok 11 - Category is isolation
ok 12 - duration is long
ok 13 - Did not see directive after code line
ok 14 - No SHBANG switches
ok 15 - got shbang
ok 16 - Got queue item data
1..16
}
ok 14 - extra_comments {
ok 1 - Timeouts turned off
ok 2 - Timeouts turned off even with default 1
ok 3 - Forking is ok
ok 4 - Checking fork with different default
ok 5 - Preload is ok
ok 6 - Checking preload with different default
ok 7 - Use isolation
ok 8 - Use isolation even with a default of false
ok 9 - Use stream
ok 10 - no stream with a default of false
ok 11 - Category is isolation
ok 12 - Duration is long
ok 13 - No SHBANG switches
ok 14 - got shbang
ok 15 - Got queue item data
1..15
}
ok 15 - conflicts {
ok 1 - 1 conflict line is reflected as an array
ok 2 - 1 conflict line with 2 conflict categories
ok 3 - 2 conflict lines with some comments on one of them
ok 4 - Duplicate conflict lines only lead to 2 conflict items.
1..4
}
ok 16 - binary {
ok 1 - File is binary
ok 2 - File must be executable
ok 3 - No SHBANG switches
ok 4 - No shbang
ok 5 - Got queue item data
1..5
}
ok 17 - not_perl {
ok 1 - No SHBANG switches
ok 2 - Non-perl shbang
ok 3 - Got queue item data
1..3
}
ok 18 - not_env_perl {
ok 1 - No SHBANG switches
ok 2 - Non-perl shbang
ok 3 - Got queue item data
1..3
}
ok 19 - smoke {
ok 1 - Turned smoke on
ok 2 - Got queue item data
ok 3 - Turned smoke on
1..3
}
ok 20 - smoke {
ok 1 - Enabled retry
ok 2 - not isolated
ok 3 - Enabled retry, value of 5 results in '6' because of initial try
ok 4 - not isolated
ok 5 - Enabled retry
ok 6 - isolated retry
ok 7 - Enabled retry, 1 initital + 3 retries
ok 8 - isolated retry
ok 9 - Retry set to 0
ok 10 - not isolated
1..10
}
1..20
ok
t/unit/Test2/Harness/Util.t ...................................
# Seeded srand with seed '20250523' from local date.
ok 1 - use Test2::Harness::Util;
ok 2 - Imported symbols: fqmod, maybe_open_file, maybe_read_fi...
ok 3 - Read file (only checking first line)
ok 4 - Exception thrown when read_file used on non-existing file
ok 5 - maybe_read_file reads file when it exists
ok 6 - maybe_read_file is undef when file does not exist
ok 7 - opened file
ok 8 - Can read from file, default mode is 'read'
ok 9 - opened /dev/null for writing
ok 10 - printed to /dev/null
ok 11 - wrote and returned content (/dev/null)
ok 12 - maybe_open_file is undef when file does not exist
ok 13 - fqmod on postfix
ok 14 - fqmod on longer postfix
ok 15 - fqmod on fq
ok 16 - fqmod on longer fq
ok 17 - read data from file
ok 18 - Matching filenames
ok 19 - hard link
ok 20 - soft link
ok 21 - Different files
1..21
ok
t/unit/Test2/Harness/Util/File.t ..............................
# Seeded srand with seed '20250523' from local date.
ok 1 - use Test2::Harness::Util::File;
ok 2 - Test2::Harness::Util::File->can(...)
ok 3 - Must provide the 'name' attribute
ok 4 - saved fh
ok 5 - fh was set to non-blocking
ok 6 - This file exists
ok 7 - The file does not exist
ok 8 - base class decode does nothing
ok 9 - base class encode does nothing
ok 10 - opened file (for reading)
ok 11 - Cannot open file (for reading)
ok 12 - Can read file (using maybe_read)
ok 13 - maybe_read returns undef for non-existant file
ok 14 - Can read file
ok 15 - read() dies on missing file
ok 16 - Can generate an FH
ok 17 - FH is remembered
ok 18 - FH is non-blocking
ok 19 - return undef for missing file
ok 20 - can set done
ok 21 - removed fh
ok 22 - cleared done flag
ok 23 - cannot read lines from missing file
ok 24 - Got first line
ok 25 - no line to read yet
ok 26 - Got final line with no terminator
ok 27 - Got first line again after reset
1..27
ok
t/unit/Test2/Harness/Util/File/JSON.t .........................
# Seeded srand with seed '20250523' from local date.
ok 1 - use Test2::Harness::Util::File::JSON;
ok 2 - Test2::Harness::Util::File::JSON->isa('Test2::Harness::Util::File')
ok 3 - decode will decode json
ok 4 - encode will encode json
ok 5 - Got expected exception for reset()
ok 6 - Got expected exception for read_line()
1..6
ok
t/unit/Test2/Harness/Util/File/JSONL.t ........................
# Seeded srand with seed '20250523' from local date.
ok 1 - use Test2::Harness::Util::File::JSONL;
ok 2 - Test2::Harness::Util::File::JSONL->isa('Test2::Harness::Util::File')
ok 3 - Test2::Harness::Util::File::JSONL->isa('Test2::Harness::Util::File::Stream')
ok 4 - decode will decode json
ok 5 - encode will encode json and append a newline
1..5
ok
t/unit/Test2/Harness/Util/File/Stream.t .......................
# Seeded srand with seed '20250523' from local date.
ok 1 - use Test2::Harness::Util::File::Stream;
ok 2 - New instance
ok 3 - file written as expected
ok 4 - got first line
ok 5 - Got unseen completed lines, but not incomplete line
ok 6 - no new lines are ready
ok 7 - Read gets lines
ok 8 - Read sees the new lines
ok 9 - Poll sees new line after a read
ok 10 - read_line moves to the next line
ok 11 - no new lines are ready
ok 12 - no new lines are ready
ok 13 - got unterminated line after 'done' was set
ok 14 - read all lines but the last unterminated one
ok 15 - poll all lines but the last unterminated one
ok 16 - got unterminated line after 'done' was set
ok 17 - Was able to seek past the first item
1..17
ok
t/unit/Test2/Harness/Util/File/Value.t ........................
# Seeded srand with seed '20250523' from local date.
ok 1 - use Test2::Harness::Util::File::Value;
ok 2 - Test2::Harness::Util::File::Value->isa('Test2::Harness::Util::File')
ok 3 - trailing newline was removed from the value
ok 4 - got line, no newline
1..4
ok
t/unit/Test2/Harness/Util/JSON.t ..............................
# Seeded srand with seed '20250523' from local date.
ok 1 - use Test2::Harness::Util::JSON;
ok 2 - Imported symbols: JSON, encode_json, decode_json, encod...
ok 3 - Have JSON constant
ok 4 - JSON returns a class (JSON::PP)
ok 5 - Round Trip encode_json+decode
ok 6 - undef/null round-trip encode_json+decode
ok 7 - Round Trip encode_pretty_json+decode
ok 8 - undef/null round-trip encode_pretty_json+decode
ok 9 - Round Trip encode_canon_json+decode
ok 10 - undef/null round-trip encode_canon_json+decode
1..10
ok
t/unit/Test2/Harness/Util/Term.t ..............................
# Seeded srand with seed '20250523' from local date.
ok 1 - use Test2::Harness::Util::Term;
ok 2 - Imported symbol: USE_ANSI_COLOR
ok 3 - USE_ANSI_COLOR returns true or false
1..3
ok
t/unit/Test2/Tools/HarnessTester.t ............................
# Seeded srand with seed '20250523' from local date.
ok 1 - Imported symbol: summarize_events
ok 2 - Failure, assertion count, plan
ok 3 - pass, assertion count, plan
1..3
ok
t/yath_script.t ...............................................
ok 1 - Found .yath.rc in a higher dir
ok 2 - Found .yath.user.rc in the current dir
ok 3 - Got pre-args from all config files
ok 4 - Prepended args to @ARGV
ok 5 - Parsed all command args properly
ok 6 - Will come back and clean these later
ok 7 - Modified @ARGV
ok 8 - Got expected libs
ok 9 - Got expected devlibs
ok 10 - prepended libs to @INC
ok 11 - Set no plugins
ok 12 - Saw each arg as many times as we expected (including the reset mid-way wiping previously seen out)
ok 13 - Modified @ARGV
ok 14 - Got expected libs
ok 15 - prepended libs to @INC
ok 16 - Saw each arg as many times as we expected
ok 17 - Did not load anything.
ok 18 - Cleaned up @INC
ok 19 - Cleaned up @DEVLIBS
ok 20 - Cleaned up %CONFIG
ok 21 - Set SCRIPT
ok 22 - Valid path for script
ok 23 - Did not exec
ok 24 - Did not die
ok 25 - Did not warn
ok 26 - Initial script is old
ok 27 - exec called new yath
ok 28 - Died when exec failed
ok 29 - Warned about the exec
ok 30 - App::Yath=HASH->isa('App::Yath')
ok 31 - Got correct symbol
ok 32 - Used ARGV
ok 33 - Used CONFIG
1..33
ok
Test Summary Report
-------------------
t/integration/failed.t (Wstat: 256 (exited 1) Tests: 1 Failed: 1)
Failed test: 1
Non-zero exit status: 1
t/integration/reload.t (Wstat: 256 (exited 1) Tests: 2 Failed: 1)
Failed test: 2
Non-zero exit status: 1
Files=61, Tests=949, 187 wallclock secs ( 0.14 usr 0.07 sys + 71.43 cusr 51.61 csys = 123.25 CPU)
Result: FAIL
Failed 2/61 test programs. 2/949 subtests failed.
make[2]: *** [Makefile:1366: test_dynamic] Error 255
make[2]: Leaving directory '/<<PKGBUILDDIR>>'
dh_auto_test: error: make -j1 test TEST_VERBOSE=1 returned exit code 2
make[1]: *** [debian/rules:12: override_dh_auto_test] Error 25
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
make: *** [debian/rules:6: binary] Error 2
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
--------------------------------------------------------------------------------
The above is just how the build ends and not necessarily the most relevant part.
If required, the full build log is available here:
https://people.debian.org/~sanvila/build-logs/202505/
(I've actually put several build logs there).
About the archive rebuild: The build was made on virtual machines from AWS,
using sbuild and a reduced chroot with only build-essential packages.
If you could not reproduce the bug please contact me privately, as I
am willing to provide ssh access to a virtual machine where the bug is
fully reproducible.
If this is really a bug in one of the build-depends, please use
reassign and add an affects on src:libtest2-harness-perl, so that this is still
visible in the BTS web page for this package.
Notes:
- It does not always fail to build.
- The approximate failure rates I get are 50% on machines with 1 cpu
and 17% on machines with 2 cpus. Therefore, I recommend that you
try GRUB_CMDLINE_LINUX="nr_cpus=1" to reproduce, and if it
does not work, I can always offer a test VM.
- See also:
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/libtest2-harness-perl.html
Thanks.
More information about the pkg-perl-maintainers
mailing list