Bug#1093484: diffoscope: crash when processing IDEA CE distributions

Julien Plissonneau Duquène sre4ever at free.fr
Sun Jan 19 08:24:23 GMT 2025


Package: diffoscope
Version: 285
Severity: normal
X-Debbugs-Cc: sre4ever at free.fr

Dear Maintainers,

While trying to compare two versions of the IDEA CE upstream distribution
diffoscope failed with the following errors:

============================================================================
$ diffoscope --html-dir t ideaIC-2024.3.1.1.tar.gz ideaIC-2024.3.2.tar.gz
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 334, in diff
    with FIFOFeeder(feeder1, fifo1_path) as fifo1, FIFOFeeder(
                                                   ^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 225, in
__exit__
    self.join()
  File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 258, in join
    raise self._exception
  File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 249, in run
    end_nl = self.feeder(fifo)
             ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/feeders.py", line 126, in
feeder
    end_nl = feeder(out_file)
             ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/feeders.py", line 81, in
feeder
    out = filter_reader(buf, filter)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/feeders.py", line 47, in
filter_reader
    buf = additional_filter(buf)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/zip.py", line 57,
in filter
    m = self.re_strip_path.match(line.decode("utf-8"))
                                 ^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf7 in position 1: invalid
start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 780, in main
    sys.exit(run_diffoscope(parsed_args))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 720, in
run_diffoscope
    difference = compare_root_paths(path1, path2)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/compare.py", line 69, in
compare_root_paths
    difference = compare_files(file1, file2)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/compare.py", line 149, in compare_files
    return file1.compare(file2, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 536, in compare
    difference = self._compare_using_details(other, source)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 471, in _compare_using_details
    details.extend(
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/container.py", line 201, in compare_pair
    difference = compare_files(
                 ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/compare.py", line 149, in compare_files
    return file1.compare(file2, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 536, in compare
    difference = self._compare_using_details(other, source)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 471, in _compare_using_details
    details.extend(
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/container.py", line 201, in compare_pair
    difference = compare_files(
                 ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-
packages/diffoscope/comparators/utils/compare.py", line 149, in compare_files
    return file1.compare(file2, source)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/zip.py", line
267, in compare
    x = super().compare(other, source)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/zip.py", line
234, in compare
    x = super().compare(other, source)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 536, in compare
    difference = self._compare_using_details(other, source)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/utils/file.py",
line 437, in _compare_using_details
    details.extend(self.compare_details(other, source))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/zip.py", line
284, in compare_details
    differences.extend(zipinfo_differences(self, other))
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/zip.py", line
128, in zipinfo_differences
    verbose = Difference.from_operation(ZipinfoVerbose, file.path, other.path)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 267, in
from_operation
    return Difference.from_operation_exc(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 316, in
from_operation_exc
    difference = Difference.from_feeder(
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/difference.py", line 214, in
from_feeder
    unified_diff = diff(feeder1, feeder2)
                   ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 334, in diff
    with FIFOFeeder(feeder1, fifo1_path) as fifo1, FIFOFeeder(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 225, in
__exit__
    self.join()
  File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 258, in join
    raise self._exception
  File "/usr/lib/python3/dist-packages/diffoscope/diff.py", line 249, in run
    end_nl = self.feeder(fifo)
             ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/feeders.py", line 126, in
feeder
    end_nl = feeder(out_file)
             ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/feeders.py", line 81, in
feeder
    out = filter_reader(buf, filter)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/feeders.py", line 47, in
filter_reader
    buf = additional_filter(buf)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/diffoscope/comparators/zip.py", line 57,
in filter
    m = self.re_strip_path.match(line.decode("utf-8"))
                                 ^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf7 in position 1: invalid
start byte

============================================================================

Download URLs are:

- https://download.jetbrains.com/idea/ideaIC-2024.3.1.1.tar.gz (887 MiB)
- https://download.jetbrains.com/idea/ideaIC-2024.3.2.tar.gz (944 MiB)

SHA256 sums:

b183b126de2cd457475eea184874b5da2fa33ba5ae2ff874bdc8c1d534156428
*ideaIC-2024.3.1.1.tar.gz
ad9d587b704806d9a21dfebe5c51415df94699bba958d7133763cfd56934ced8
*ideaIC-2024.3.2.tar.gz

Best regards,

--
Julien Plissonneau Duquène


-- System Information:
Debian Release: trixie/sid
  APT prefers testing
  APT policy: (990, 'testing'), (20, 'unstable'), (10, 'experimental'), (5, 'stable-updates'), (5, 'stable-security'), (5, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 6.11.10-amd64 (SMP w/12 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages diffoscope depends on:
ii  diffoscope-minimal  285

Versions of packages diffoscope recommends:
ii  7zip                             24.09+dfsg-3
ii  aapt                             1:14~beta1-3
ii  abootimg                         0.6-1.1
ii  acl                              2.3.2-2+b1
ii  androguard                       3.4.0~a1-16
ii  apksigcopier                     1.1.1-1
ii  apksigner                        31.0.2-2
ii  apktool                          2.7.0+dfsg-7
ii  binutils-multiarch               2.43.50.20250108-1
ii  bzip2                            1.0.8-6
ii  caca-utils                       0.99.beta20-5
ii  colord                           1.4.7-1+b2
ii  coreboot-utils                   24.08+dfsg-2
ii  db-util                          5.3.4
ii  default-jdk [java-sdk]           2:1.21-76
ii  default-jdk-headless             2:1.21-76
ii  device-tree-compiler             1.7.2-2
ii  dexdump                          14.0.0+r15-4
ii  docx2txt                         1.4-5
ii  e2fsprogs                        1.47.2-1
ii  enjarify                         1:1.0.3-5
ii  ffmpeg                           7:7.1-3+b1
ii  fontforge-extras                 1:20230101~dfsg-4
ii  fonttools                        4.55.3-1
ii  fp-utils                         3.2.2+dfsg-35
ii  fp-utils-3.2.2 [fp-utils]        3.2.2+dfsg-35
ii  genisoimage                      9:1.1.11-4
ii  gettext                          0.22.5-4
ii  ghc                              9.6.6-3
ii  ghostscript                      10.04.0~dfsg-2+b1
ii  giflib-tools                     5.2.2-1+b1
ii  gnumeric                         1.12.57-1.1+b1
ii  gnupg-utils                      2.2.46-1
ii  gpg                              2.2.46-1
ii  hdf5-tools                       1.14.5+repack-3
ii  html2text                        2.2.3-3
ii  imagemagick                      8:7.1.1.43+dfsg1-1
ii  imagemagick-7.q16 [imagemagick]  8:7.1.1.43+dfsg1-1
ii  jsbeautifier                     1.15.1-1
ii  libarchive-tools                 3.7.4-1.1
ii  libxmlb-utils                    0.3.21-1
ii  llvm                             1:19.0-63
ii  lz4                              1.9.4-4
ii  lzip                             1.25~rc1-1
ii  mono-utils                       6.12.0.199+dfsg-2.1
ii  ocaml-nox                        5.2.0-3
ii  odt2txt                          0.5-7
ii  oggvideotools                    0.9.1-7
ii  openjdk-11-jdk [java-sdk]        11.0.25+9-1
ii  openjdk-21-jdk [java-sdk]        21.0.5+11-1
ii  openjdk-8-jdk [java-sdk]         8u432-b06-2
ii  openssh-client                   1:9.9p1-3
ii  openssl                          3.4.0-2
ii  perl                             5.40.0-8
ii  pgpdump                          0.36-1
ii  poppler-utils                    24.08.0-4
ii  procyon-decompiler               0.6.0-3
ii  python3-argcomplete              3.5.3-1
ii  python3-binwalk                  2.4.3+dfsg1-1
ii  python3-debian                   0.1.49
ii  python3-defusedxml               0.7.1-3
ii  python3-guestfs                  1:1.52.2-7+b2
ii  python3-jsondiff                 2.1.2-1
ii  python3-pdfminer                 20221105+dfsg-1
ii  python3-progressbar              2.5-4
ii  python3-pypdf                    4.3.1-1
ii  python3-pyxattr                  0.8.1-1+b5
ii  python3-rpm                      4.20.0+dfsg-3
ii  python3-tlsh                     3.4.4+20151206-1.4+b8
ii  r-base-core                      4.4.2-1
ii  radare2                          5.9.8+dfsg-2
ii  rpm2cpio                         4.20.0+dfsg-3
ii  sng                              1.1.0-4+b1
ii  sqlite3                          3.46.1-1
ii  squashfs-tools                   1:4.6.1-1
ii  systemd-ukify                    257.2-1
ii  tcpdump                          4.99.5-1
ii  u-boot-tools                     2024.01+dfsg-7
ii  unzip                            6.0-28
ii  wabt                             1.0.34+dsfg2+~cs1.0.32-1+b1
ii  xmlbeans                         4.0.0-2
ii  xxd                              2:9.1.0967-2
ii  xz-utils                         5.6.3-1+b1
ii  zip                              3.0-14
ii  zstd                             1.5.6+dfsg-2

Versions of packages diffoscope suggests:
ii  libjs-jquery  3.6.1+dfsg+~3.5.14-1

-- no debconf information


More information about the Reproducible-builds mailing list