Bug#991059: diffoscope: out-of-memory
rclobus at rclobus.nl
Wed Aug 11 10:49:46 BST 2021
On 15/07/2021 18:41, Mattia Rizzolo wrote:
> On Wed, Jul 14, 2021 at 08:31:48PM +0200, Roland Clobus wrote:
>> The difference between these two files is located inside a squashfs
>> file, which is inside the iso file.
>> During the invocation of diffoscope, diffoscope needs lots of memory
>> (>32GB) and free space on /tmp (>32GB but <48GB).
I've got the 2 ISO files that were used in a Jenkins run now, each 2.6GB.
In attempting to reproduce the case, I've mounted /tmp to a file instead
of using tmpfs and I've booted with 'single'. My computer has 32GB
memory, about 200MB is in use by the OS.
When I'm running diffoscope as root, I get the following metrics
(obtained by polling every 5 seconds):
21935816 1k blocks on /tmp (about 22GB)
1010 MiB memory at the peak
Needed time: 120 minutes +/ 10
However, when I run the same command as a regular user, I get an OOM
after about 5 minutes. At that time, the first squashfs image (2.3GB) is
completely decompressed to disc (8.0GB), and xxd is running.
The output of diffoscope with --debug:
2021-08-11 09:15:48 D: diffoscope.comparators.utils.file: Instantiating
a squashfs.SquashfsContainer for live/filesystem.squashfs
2021-08-11 09:15:48 D: diffoscope.comparators.squashfs: Extracting
2021-08-11 09:15:48 D: diffoscope.comparators.utils.command: Calling
external command: unsquashfs -n -f -no -li -d .
2021-08-11 09:16:55 D: diffoscope.comparators.utils.command: Executing
2021-08-11 09:18:40 D: diffoscope.comparators.utils.command: Executing
So it appears to me that different code is activated for regular users
I hope this report helps in finding/fixing the issue.
With kind regards,
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 833 bytes
Desc: OpenPGP digital signature
More information about the Reproducible-builds