Bug#1142815: libipc-shareable-perl: autopkgtest uses insane amount of memory on loong64

[Edmund Lodewijks]

On Thu, 27 Aug 2026 14:52:41 +0200 gregor herrmann <gregoa at debian.org> 
wrote:
> (And I hope smeone else will come along and "just fix" this package 
> :))

Hi all,

Okay, I had a stab at this. I looked at IPC-Shareable-1.19/t/47-seg_size.t.

My background is rather in C, and I have never written any Perl, so for 
the patch, I will happily leave that to the Perl devs here. Therefore, I 
am afraid, no "just fix" solution; but still the cause and a write-up 
for a patch.

## The Root Cause

The test has a "beyond RAM limits" block that passes `limit => 0` to 
bypass internal module checks, and intentionally requests an 
"impossible" 999,999,999,999-byte (~1 TB) segment, expecting the OS to 
refuse it.

- On non-Linux systems (macOS, BSD), this works, because they enforce 
small, strict default SHMMAX ceilings (e.g., 4 MB to 512 MB).
- On Linux, the default `kernel.shmmax` is effectively unlimited 
(`ULONG_MAX - 2**24` - check `/proc/sys/kernel/shmmax`). Therefore, 
`shmget()` falls through to the standard overcommit heuristic in 
`__vm_enough_memory()`, which only rejects segments that exceed MemTotal 
+ SwapTotal.
- On large infrastructure nodes where RAM + swap exceeds 1 TB, the 
kernel grants the segment lazily. When `SharedMem.pm` subsequently 
performs a `shmread()` to inspect the segment, it forces two massive 
physical allocations simultaneously: a 1 TB Perl scalar heap allocation 
for the read buffer, and the physical materialisation of the 1 TB 
zero-filled shared memory segment via page faults. This results in an 
immediate 2 TB memory footprint spike, causing the test runner to be killed.

Paul's numbers confirm this.

Paul reported a virtual size of 1,953,147,760 KiB for `/usr/bin/perl 
t/65-seg_size.t` on line 601. When converted, that is exactly 
2,000,023,306,240 bytes. Against the 1 TB target request, this maps 
almost cleanly to a 2.0000x multiplier. The remaining ~22 MB is also to 
be found in Paul's message: `/usr/bin/perl /usr/bin/prove` on line 463 
shows a virtual size of 22240 KiB = 22.19 MiB.

## Proposed Fix Architecture

Rather than relying on a hardcoded "impossible" size that modern server 
hardware can easily accommodate, the test script (t/47-seg_size.t) 
should calculate a dynamic refusal ceiling at runtime:

1. Keep a baseline fallback: Define a static constant of 999999999999 
bytes (~1 TB) as the default choice for non-Linux architectures.
2. Handle a lowered Linux shmmax (if defined): On Linux, check 
`/proc/sys/kernel/shmmax`. If a system administrator has manually set it 
to a value lower than 1 TB, keep the 1 TB test size, as the OS will 
already safely reject it.
3. Account for Overcommit Mode 1: Check 
`/proc/sys/vm/overcommit_memory`. If it is set to 1 (always overcommit), 
the kernel will never reject an allocation at creation time. In this 
case, cleanly skip the two test assertions to prevent a guaranteed 
out-of-memory crash.
4. Compute a dynamic ceiling via `meminfo`: If overcommit is in standard 
mode, parse `/proc/meminfo` to read MemTotal and SwapTotal. Calculate a 
new target size equivalent to MemTotal + SwapTotal + 1 GB (adding 1 GB 
to safely clear page-alignment rounding bounds). Use this computed size 
for the tie operation to guarantee an immediate application-layer or 
kernel-level refusal without initialising any real memory blocks.
5. Enforce defensive container fallbacks: Ensure that all file-read 
operations on `/proc` files fail gracefully (returning the 1 TB baseline 
fallback) if executed inside locked-down sandbox environments, Distrobox 
environments, or strict container runtimes where `/proc` access is 
restricted.

This runtime logic completely avoids hardcoding architecture-specific 
blacklists, cleanly accommodates modern high-memory hardware 
configurations, and prevents sudden worker terminations.

     Edmund


-- 
Edmund Lodewijks <edmund at proteamail.com>
TZ: UTC+2 / GMT+2

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/pkg-perl-maintainers/attachments/20260827/ccb57177/attachment.sig>