[Reproducible-builds] Bug#780761: debbindiff fails noisily when run over perversely recursive input files

[Daniel Kahn Gillmor]

Package: debbindiff
Version: 9
Severity: wishlist

I'm attaching two ugly fake little .debs that look roughly the same
from the outside.  they each ship only one file in data.tar.gz, which
is r.zip.

in both cases, r.zip is a nasty file that unpacks to show another
file, r.zip, which happens to be identical to the original file [0].

however, the r.zip files in each .deb differ from each other a little
bit.

the result is a huge python backtrace that ultimately ends with:

  File "/usr/lib/python2.7/dist-packages/debbindiff/comparators/zip.py", line 61, in compare_zip_files
    source=name))
  File "/usr/lib/python2.7/dist-packages/debbindiff/comparators/__init__.py", line 119, in compare_files
    return comparator(path1, path2, source)
  File "/usr/lib/python2.7/dist-packages/debbindiff/comparators/utils.py", line 55, in with_fallback
    inside_differences = original_function(path1, path2, source)
  File "/usr/lib/python2.7/dist-packages/debbindiff/comparators/zip.py", line 43, in compare_zip_files
    with ZipFile(path1, 'r') as zip1:
  File "/usr/lib/python2.7/zipfile.py", line 770, in __init__
    self._RealGetContents()
  File "/usr/lib/python2.7/zipfile.py", line 844, in _RealGetContents
    x = ZipInfo(filename)
RuntimeError: maximum recursion depth exceeded


Perhaps leaving debbindiff to the mercies of the python recursion
limits is the way to go here -- at least it's a relatively quick
failure.  But it's conceivable that we want to do something marginally
cleaner.

Some options might be:

 0) limit the unpacking recursion ourselves (mayve 20 levels?), and
    having a graceful termination suggesting that the packages are
    likely to be bogus?

 1) caching the digest of every archive unpacked on the stack, and
    bailing gracefully when we find another internal archive with a
    digest already unpacked.  (maybe this doesn't need to be an error
    for debbindiff -- could we just stop the nested recursion there
    and treat the archive as a non-archive?)

or we could do both of the above, i guess.  I'm open to other
suggestions.

    --dkg

[0] hat tip to http://research.swtch.com/zip for the construction

-- System Information:
Debian Release: 8.0
  APT prefers testing
  APT policy: (500, 'testing'), (200, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages debbindiff depends on:
ii  binutils-multiarch  2.25-5
ii  bzip2               1.0.6-7+b2
ii  cpio                2.11+dfsg-4.1
ii  diffutils           1:3.3-1+b1
ii  file                1:5.22+15-2
ii  fontforge-extras    0.3-4
ii  gettext             0.19.3-2
ii  ghc                 7.6.3-20
ii  gnupg               1.4.19-1
ii  pdftk               2.02-2
ii  poppler-utils       0.26.5-2
ii  python              2.7.8-4
ii  python-debian       0.1.25
ii  python-magic        1:5.22+15-2
ii  python-rpm          4.11.3-1.1
ii  rpm2cpio            4.11.3-1.1
ii  sng                 1.0.2-7
ii  unzip               6.0-16
ii  vim                 2:7.4.488-4
ii  vim-common          2:7.4.488-4
ii  xz-utils            5.1.1alpha+20120614-2+b3

debbindiff recommends no packages.

debbindiff suggests no packages.

-- debconf-show failed
-------------- next part --------------
A non-text attachment was scrubbed...
Name: turtles_1_all.deb
Type: application/vnd.debian.binary-package
Size: 644 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/reproducible-builds/attachments/20150318/16edd793/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: turtles_2_all.deb
Type: application/vnd.debian.binary-package
Size: 644 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/reproducible-builds/attachments/20150318/16edd793/attachment-0001.bin>