Bug#1035983: libsoup3 (and libsoup2) autopkgtests are flaky: Address already in use: AH00072: make_sock: could not bind to address 127.0.0.1:47524

[Simon McVittie]

On Fri, 16 May 2025 at 12:45:43 +0200, Santiago Vila wrote:
>Regarding the flakiness itself, I get a failure rate around 20%
>on machines with 1 CPU and 30% on machines with 2 CPUs. This is
>greater than the reference thresholds given by Paul in one
>of the gcr bugs.
>
>I'd like to propose a patch, but the tests which fail are
>different every time. On a sample of 200 build tries on
>different machines, I get the following failures these many times:
>
>  26 multithread-test
>  23 proxy-test
>  22 range-test
>  22 connection-test
>  22 auth-test
>   6 server-test
>   1 timeout-test
>   1 hsts-test

Is this still the same failure mode described in the bug title, with 
"Address already in use" and "could not bind to address ..." being 
reported by Apache?

Last time I looked at the libsoup* test suite, the actual tests were 
each reasonably reliable, but the reliability issue was with their 
setup/teardown. They run a temporary Apache web server, in order to 
have a realistic server to test against. I think what's happening is 
that sometimes, the web server port from one test (let's say test number 
5) is still considered by the kernel to be in use by the time we reach 
the setup stage of the next test (let's say test number 6).

As a result, the Apache for test number 6 can't listen on the port it 
has been configured to use, and testing fails at that point. This is 
rare on a per-test basis, therefore difficult to reproduce on-demand - 
but running the whole test suite involves several setup/teardown cycles, 
resulting in a higher failure rate for the test suite as a whole. For 
example if you're seeing a 30% failure rate, that might be more like a 
2% failure rate for each of 15 test executables, or perhaps even a 0.2% 
failure rate for each of 150 smaller test-cases.

If that's still what is happening, then it's expected that you will see 
failures in different tests (and even in different test-cases within 
those larger tests) on different occasions.

Unfortunately, if that's the case, then skipping any specific test-case 
is not going to be a viable workaround, because it's the common 
setup/teardown done for each test-case that is the problem.

If it's possible to configure Apache to set options like SO_REUSEADDR 
and/or SO_REUSEPORT then that might help (but I don't know whether 
that's possible).

Or if it's possible to make the test suite use a different port for each 
test then that might help (but I don't know whether that will be 
feasible).

     smcv