Bug#991059: diffoscope: subprocess.CalledProcessError: Command unsquashfs... returned non-zero exit status 1

[Holger Levsen]

Package: diffoscope
Version: 177
Severity: normal
x-debbugs-cc: Roland Clobus <rclobus at rclobus.nl>

Dear Maintainer,

https://jenkins.debian.net/job/reproducible_debian_live_build_cinnamon_bullseye/lastFailedBuild/consoleFull
shows a failure to run diffoscope on a cinnamon libe-build hybris.iso:

+ timeout 30m nice schroot --directory /srv/reproducible-results/live-build-cinnamon-Xj14Z0Pu -c source:jenkins-reproducible-unstable-diffoscope diffoscope -- --html /srv/reproducible-results/live-build-cinnamon-Xj14Z0Pu/live-build/cinnamon/live-image-amd64.hybrid.iso.html /srv/reproducible-results/live-build-cinnamon-Xj14Z0Pu/b1/live-build/cinnamon/live-image-amd64.hybrid.iso /srv/reproducible-results/live-build-cinnamon-Xj14Z0Pu/b2/live-build/cinnamon/live-image-amd64.hybrid.iso
+ RESULT=2
++ grep '^E: 15binfmt: update-binfmts: unable to open' /srv/reproducible-results/live-build-cinnamon-Xj14Z0Pu/tmp.VaTe8khm5d
++ true
+ LOG_RESULT=
+ '[' '!' -z '' ']'
+ true
+ set -e
+ cat /srv/reproducible-results/live-build-cinnamon-Xj14Z0Pu/tmp.VaTe8khm5d
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 494, in compare
    difference = self._compare_using_details(other, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 429, in _compare_using_details
    details.extend(
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 130, in comparisons
    my_members = OrderedDict(self.get_adjusted_members_sizes())
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 122, in get_adjusted_members_sizes
    for name, member in self.get_adjusted_members():
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 78, in get_filtered_members
    for name in filter_excludes(self.get_member_names()):
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/squashfs.py", line 250, in get_member_names
    self.ensure_unpacked()
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/squashfs.py", line 263, in ensure_unpacked
    output = our_check_output(
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/command.py", line 117, in our_check_output
    return subprocess.check_output(cmd, *args, **kwargs)
  File "/usr/lib/python3.9/subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('unsquashfs', '-n', '-f', '-no', '-li', '-d', '.', '/tmp/diffoscope_tj0g1di9_cinnamon/tmp_ss17mt0LibarchiveContainerWithFilelist/0/891.squashfs')' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 746, in main
    sys.exit(run_diffoscope(parsed_args))
  File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 700, in run_diffoscope
    difference = compare_root_paths(path1, path2)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 69, in compare_root_paths
    difference = compare_files(file1, file2)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 125, in compare_files
    return file1.compare(file2, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 494, in compare
    difference = self._compare_using_details(other, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 430, in _compare_using_details
    self.as_container.compare(
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/libarchive.py", line 366, in compare
    differences.extend(super().compare(other, **kwargs))
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 191, in compare_pair
    difference = compare_files(
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 125, in compare_files
    return file1.compare(file2, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 515, in compare
    difference = self.compare_bytes(other, source=source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 382, in compare_bytes
    return compare_binary_files(self, other, source)
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 151, in compare_binary_files
    return Difference.from_operation(
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 269, in from_operation
    return Difference.from_operation_exc(
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 290, in from_operation_exc
    feeder1, operation1, excluded1 = operation_and_feeder(path1)
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 287, in operation_and_feeder
    operation.start()
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/command.py", line 45, in start
    self._process = subprocess.run(
  File "/usr/lib/python3.9/subprocess.py", line 507, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "/usr/lib/python3.9/subprocess.py", line 1134, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.9/subprocess.py", line 2001, in _communicate
    data = os.read(key.fd, 32768)
MemoryError

Sadly I don't have those .iso files available but I suppose we provide them if needed.


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

Words may inspire but only action creates change.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20210713/a1a74354/attachment.sig>