[Debian-med-packaging] Bug#1053509: shasta: autopkgtest regression on arm64: is memory suddenly not enough?

[Étienne Mollier]

Hi Paul,
> 1) why does it now suddenly start to (nearly always) fail across the 
> board on arm64 (in Debian, Ubuntu still seems fine), without changes to 
> the infrastructure that I know of?

I'm afraid I'm not sure what is up with shasta eating up more
memory on arm64 hosts of CI infrastructure.  What I can see from
my end is that the test roughly requires 8GiB of anonymous
memory to map for doing its job.  Except that, this is already
the case for shasta in bookworm running on bookworm kernel, so
that doesn't look to be a regression per se.

> 2) do you also believe this is related to memory consumption?

The problem you mentioned, where shasta explicitly gives up when
running into memory limits, is reproducible when I disable the
swap on an 8GiB machine that I have at hand.  I attempted to
play with /proc/sys/vm/overcommmit_* settings, but my swap at t
time was too big (10GiB) to give me the granularity necessary to
check whether I could get somewhere with improper overcommit
memory tuning.  In any cases, the "Killed" status suggest
overcommit is active (or heuristic) on your end for at least
some of the hosts.

Per chance, could you double check the memory settings on the CI
hosts, just in case, to make sure that the swap didn't drop off
the machine?  Or maybe check for memory overcommit settings
inconsistencies?

Currently readable test logs suggest that:

  * ci-worker-arm64-10 met memory requirements in November,
  * ci-worker-arm64-07 did not meet requirements in October,
  * ci-worker-arm64-08 did not meet requirements in October,
  * ci-worker-arm64-03 did not meet requirements in October.

So this may already be resolved, in case you changed something
in between.

> 3) If 2 == yes, what are the memory requirements for the test? The test 
> *could* test for that before it starts and bail out (restriction: 
> skippable with exit code 77 [2]) if the amount of memory available is 
> too small.

It shouldn't hurt I guess.  I think I can bolt something reading
the memory commit capacity and usage in /proc/meminfo at the
beginning of the test, and skip the run if the testbed couldn't
meet the memory requirement for whatever reason.  Note this may
involve some trial and error.

Have a nice Sunday,  :)
-- 
  .''`.  Étienne Mollier <emollier at debian.org>
 : :' :  gpg: 8f91 b227 c7d6 f2b1 948c  8236 793c f67e 8f0d 11da
 `. `'   sent from /dev/pts/1, please excuse my verbosity
   `-    on air: Ghost - Avalanche
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/debian-med-packaging/attachments/20231203/f5f1cbe7/attachment.sig>