Bug#1006815: xmoto: autopkgtest often fails: Failed to connect to X-Moto: Connection refused

[Simon McVittie]

Source: xmoto
Version: 0.6.1+repack-7
Severity: serious
X-Debbugs-CC: debian-ci at lists.debian.org
User: debian-ci at lists.debian.org
Usertags: flaky

The superficial autopkgtest for xmoto seems to be failing intermittently,
especially on ppc64el (for whatever reason), which is likely to block
other packages' migration to testing.

In particular, I'm looking at this because of sdl12-compat: xmoto
currently uses libsdl1.2debian, which is the legacy SDL 1.2 package
(unmaintained upstream), but the long-term plan is for sdl12-compat
(a reimplementation of the SDL 1.2 API using SDL 2) to take over the
binary package name libsdl1.2debian.

It looks as though the test is waiting for an arbitrary 3 seconds for
the server to start listening, which is not always going to be enough.
It would be better if the test was something like this (using
approximately Python syntax here, but only as pseudocode):

     start_time = time now
     TIMEOUT = 60 seconds (or some other arbitrary semi-long time)

     while True:
         if server process has exited:
             fail the test

         if time now > start_time + TIMEOUT:
             fail the test

         try:
             connect to socket
         except ECONNREFUSED:
             sleep(1)
         except any other error:
             fail the test
         else:
             break out of this loop

     continue with the test

Or, as a stopgap, an alternative would be to mark the test as flaky
(which means its result will be ignored) and reduce the severity of this
bug report to non-RC.

Thanks,
     smcv