Bug#958271: Excessive memory use when comparing linux debug packages
Ben Hutchings
ben at decadent.org.uk
Mon Apr 20 03:31:58 BST 2020
Package: diffoscope
Version: 141
Severity: normal
Comparing two versions of linux-image-5.5.0-trunk-amd64-dbg causes
diffoscope to use up about 13 GiB of VM, and then crash because it
still wants more:
$ TMPDIR=/var/tmp diffoscope --max-diff-block-lines-saved 1000 --text dummy.diffoscope linux-image-5.6.0-trunk-amd64-dbg_5.6.4-1~exp{1,2}_amd64.deb
Traceback (most recent call last):#######| 100% ETA: 0:00:00
File "/usr/bin/diffoscope", line 36, in <module>
main()
File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 740, in main
sys.exit(run_diffoscope(parsed_args))
File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 692, in run_diffoscope
difference = compare_root_paths(path1, path2)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 74, in compare_root_paths
difference = compare_files(file1, file2)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 128, in compare_files
return file1.compare(file2, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 453, in compare
difference = self._compare_using_details(other, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 401, in _compare_using_details
other.as_container, no_recurse=no_recurse
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 195, in compare_pair
file1, file2, source=None, diff_content_only=no_recurse
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 128, in compare_files
return file1.compare(file2, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 453, in compare
difference = self._compare_using_details(other, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 401, in _compare_using_details
other.as_container, no_recurse=no_recurse
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 195, in compare_pair
file1, file2, source=None, diff_content_only=no_recurse
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 128, in compare_files
return file1.compare(file2, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 453, in compare
difference = self._compare_using_details(other, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 401, in _compare_using_details
other.as_container, no_recurse=no_recurse
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 195, in compare_pair
file1, file2, source=None, diff_content_only=no_recurse
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 128, in compare_files
return file1.compare(file2, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 453, in compare
difference = self._compare_using_details(other, source)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 371, in _compare_using_details
details.extend(self.compare_details(other, source))
File "/usr/lib/python3/dist-packages/diffoscope/comparators/elf.py", line 629, in compare_details
return _compare_elf_data(self.path, other.path)
File "/usr/lib/python3/dist-packages/diffoscope/comparators/elf.py", line 271, in _compare_elf_data
for x in list(READELF_COMMANDS) + READELF_DEBUG_DUMP_COMMANDS
File "/usr/lib/python3/dist-packages/diffoscope/comparators/elf.py", line 271, in <listcomp>
for x in list(READELF_COMMANDS) + READELF_DEBUG_DUMP_COMMANDS
File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 256, in from_command
klass, path1, path2, *args, **kwargs
File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 288, in from_command_exc
feeder1, feeder2, path1, path2, *args, **kwargs
File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 213, in from_feeder
unified_diff = diff(feeder1, feeder2)
File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 330, in diff
return run_diff(fifo1_path, fifo2_path, fifo1.end_nl_q, fifo2.end_nl_q)
File "/usr/lib/python3/dist-packages/diffoscope/tools.py", line 100, in tool_check
return fn(*args, **kwargs)
File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 179, in run_diff
p = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
File "/usr/lib/python3.7/subprocess.py", line 472, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.7/subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.7/subprocess.py", line 1453, in _execute_child
restore_signals, start_new_session, preexec_fn)
OSError: [Errno 12] Cannot allocate memory
-- System Information:
Debian Release: 10.3
APT prefers stable
APT policy: (990, 'stable'), (500, 'stable-updates'), (500, 'stable-debug'), (1, 'unstable')
Architecture: amd64 (x86_64)
Kernel: Linux 4.19.0-8-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages diffoscope depends on:
ii libpython3.5-stdlib 3.5.3-1+deb9u1
ii python3 3.7.3-1
ii python3-distro 1.3.0-1
ii python3-distutils 3.7.3-1
ii python3-libarchive-c 2.8-0.3
ii python3-magic 2:0.4.15-2
ii python3-pkg-resources 40.8.0-1
Versions of packages diffoscope recommends:
ii abootimg 0.6-1+b2
ii acl 2.2.53-4
ii apktool 2.3.4-1
ii binutils-multiarch 2.31.1-16
ii bzip2 1.0.6-9.2~deb10u1
pn caca-utils <none>
ii colord 1.4.3-4
ii db-util 5.3.1+nmu1
pn default-jdk-headless | default-jdk | java-sdk <none>
ii device-tree-compiler 1.4.7-3
ii docx2txt 1.4-1
ii e2fsprogs 1.44.5-1+deb10u3
pn enjarify <none>
ii ffmpeg 7:4.1.4-1~deb10u1
ii fontforge-extras 0.3-4
pn fp-utils <none>
ii genisoimage 9:1.1.11-3+b2
ii gettext 0.19.8.1-9
ii ghc 8.4.4+dfsg1-3
ii ghostscript 9.27~dfsg-2+deb10u3
ii giflib-tools 5.1.4-3
ii gnumeric 1.12.44-1
ii gnupg 2.2.12-1+deb10u1
ii gnupg-utils 2.2.12-1+deb10u1
pn hdf5-tools <none>
ii imagemagick 8:6.9.10.23+dfsg-2.1
ii imagemagick-6.q16 [imagemagick] 8:6.9.10.23+dfsg-2.1
ii jsbeautifier 1.6.4-7
ii libarchive-tools 3.3.3-4+deb10u1
ii llvm 1:7.0-47
ii lz4 [liblz4-tool] 1.8.3-1
ii mono-utils 5.18.0.240+dfsg-3
ii ocaml-nox 4.05.0-11
ii odt2txt 0.5-1+b2
ii oggvideotools 0.9.1-5
ii openssh-client 1:7.9p1-10+deb10u2
ii pgpdump 0.33-1
ii poppler-utils 0.71.0-5
ii procyon-decompiler 0.5.32-5
ii python3-argcomplete 1.8.1-1
ii python3-binwalk 2.1.2~git20180830+dfsg1-1
ii python3-debian 0.1.35
ii python3-defusedxml 0.5.0-2
pn python3-guestfs <none>
ii python3-jsondiff 1.1.1-2
pn python3-pdfminer <none>
ii python3-progressbar 2.5-1
ii python3-pypdf2 1.26.0-2
ii python3-pyxattr 0.6.1-1
ii python3-tlsh 3.4.4+20151206-1.1
ii r-base-core 3.5.2-1
ii rpm2cpio 4.14.2.1+dfsg1-1
ii sng 1.1.0-1+b1
ii sqlite3 3.27.2-3
ii squashfs-tools 1:4.3-12
ii tcpdump 4.9.3-1~deb10u1
ii unzip 6.0-23+deb10u1
ii vim-common 2:8.1.0875-5
pn wabt <none>
ii xmlbeans 3.0.2-1
ii xxd 2:8.1.0875-5
ii xz-utils 5.2.4-1
ii zip 3.0-11+b1
pn zstd <none>
Versions of packages diffoscope suggests:
ii libjs-jquery 3.3.1~dfsg-3
-- no debconf information
More information about the Reproducible-builds
mailing list