diffoscope timing out.

Holger Levsen holger at layer-acht.org
Wed Nov 15 09:29:01 GMT 2023 

On Mon, Nov 06, 2023 at 10:42:05AM +0000, Chris Lamb wrote:
> Christos Zoulas wrote:
> 
> > For weeks now there is no useful output from diffoscope. Is there anything I
> > can do to help debug the issue?
> >
> > tests.reproducible-builds.org        [eeB]
> > Wed 1 Nov 23:35:09 UTC 2023 - diffoscope 251 produced no output for
> > x86_64-amd64/amd64/binary/sets/comp.tar.xz and was killed after running into
> > timeout after 30m...

these issues seem to have been resolved, 
https://jenkins.debian.net/job/reproducible_netbsd/1029/console
contains no such timeouts.
 
> We'd actually love some help debugging this issue. As you can see from
> the list archives, we have indeed been seeing this for a little while:
>   https://alioth-lists.debian.net/pipermail/reproducible-builds/Week-of-Mon-20230828/thread.html
> … as well as some threads in later weeks.

I'm not sure those issues are related, maybe or maybe not.
 
> However, are unsure whether this an issue with:
> 
> a) The system running tests.reproducible-builds.org (which, I believe,
> was upgraded around the time this issue appeared?)
> 
> b) The code in diffoscope itself (which is somewhat unlikely, as there
> have not been meaningful changes that might cause crashes or
> timeouts.)

https://jenkins.debian.net/job/reproducible_netbsd/1029/console
contains this, which IMO should not happen and is an issue in
diffoscope:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 767, in main
    sys.exit(run_diffoscope(parsed_args))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 718, in run_diffoscope
    difference = compare_root_paths(path1, path2)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 69, in compare_root_paths
    difference = compare_files(file1, file2)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 149, in compare_files
    return file1.compare(file2, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 532, in compare
    difference = self._compare_using_details(other, source)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 467, in _compare_using_details
    details.extend(
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 197, in compare_pair
    difference = compare_files(
                 ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 149, in compare_files
    return file1.compare(file2, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 616, in compare
    return self.compare_bytes(other, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 412, in compare_bytes
    return compare_binary_files(self, other, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 176, in compare_binary_files
    return Difference.from_operation(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 267, in from_operation
    return Difference.from_operation_exc(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 288, in from_operation_exc
    feeder1, operation1, excluded1 = operation_and_feeder(path1)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 285, in operation_and_feeder
    operation.start()
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/command.py", line 45, in start
    self._process = subprocess.run(
                    ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 550, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1209, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 2128, in _communicate
    data = os.read(key.fd, 32768)
           ^^^^^^^^^^^^^^^^^^^^^^
MemoryError
Fri 10 Nov 22:41:22 UTC 2023 - diffoscope 251 had trouble comparing the two builds. Please investigate x86_64-amd64/images/NetBSD-10.99.10-amd64-bios-install.img.gz
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 767, in main
    sys.exit(run_diffoscope(parsed_args))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 718, in run_diffoscope
    difference = compare_root_paths(path1, path2)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 69, in compare_root_paths
    difference = compare_files(file1, file2)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 149, in compare_files
    return file1.compare(file2, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 532, in compare
    difference = self._compare_using_details(other, source)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 467, in _compare_using_details
    details.extend(
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/container.py", line 197, in compare_pair
    difference = compare_files(
                 ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 149, in compare_files
    return file1.compare(file2, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 616, in compare
    return self.compare_bytes(other, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py", line 412, in compare_bytes
    return compare_binary_files(self, other, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/compare.py", line 176, in compare_binary_files
    return Difference.from_operation(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 267, in from_operation
    return Difference.from_operation_exc(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 288, in from_operation_exc
    feeder1, operation1, excluded1 = operation_and_feeder(path1)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 285, in operation_and_feeder
    operation.start()
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/command.py", line 45, in start
    self._process = subprocess.run(
                    ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 550, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1209, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 2128, in _communicate
    data = os.read(key.fd, 32768)
           ^^^^^^^^^^^^^^^^^^^^^^
MemoryError
Fri 10 Nov 22:43:45 UTC 2023 - diffoscope 251 had trouble comparing the two builds. Please investigate x86_64-amd64/images/NetBSD-10.99.10-amd64-install.img.gz


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

三人成虎- Three men make a tiger.
In other words, if one guy says "there's a tiger over there" you might not believe 
them, if three guys in a row all say this- you think there's a tiger there. A lie, 
repeated often enough, will be accepted as truth.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20231115/263603e1/attachment.sig>

More information about the Reproducible-builds mailing list