Bug#1029066: diffoscope: FTBFS if no internet is available (using internet connection during build)

[Chris Lamb]

Hi all,

> […]

As Mattia writes on the Salsa bug [0], I now don't think this is a
network issue. In other words, the package FTBFS regardless of whether
you have network access or not.

To make debugging this easier, I've split out the inline Python code
in c341b63a [1], and simply running the new script results in:

    $ ping -q -c1 google.com 2>&1 | grep packet
    1 packets transmitted, 1 received, 0% packet loss, time 0ms

    $ debian/tests/generate-recommends.py
    ERROR: Could not find an activated virtualenv (required).
    Traceback (most recent call last):
      File "/home/lamby/git/debian/reproducible-builds/diffoscope/debian/tests/generate-recommends.py", line 7, in <module>
        dist = meta.load(".")
               ^^^^^^^^^^^^^^
      File "/usr/lib/python3/dist-packages/pep517/meta.py", line 72, in load
        path = Path(build_as_zip(builder))
                    ^^^^^^^^^^^^^^^^^^^^^
      File "/usr/lib/python3/dist-packages/pep517/meta.py", line 59, in build_as_zip
        builder(dest=out_dir)
      File "/usr/lib/python3/dist-packages/pep517/meta.py", line 53, in build
        env.pip_install(system['requires'])
      File "/usr/lib/python3/dist-packages/pep517/envbuild.py", line 103, in pip_install
        check_call(
      File "/usr/lib/python3.11/subprocess.py", line 413, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m',
      'pip', 'install', '--ignore-installed', '--prefix',
      '/tmp/pep517-build-env-jzvg739_', 'setuptools', 'wheel']' returned
      non-zero exit status 3.

Regarding when this was introduced, I think a confounding factor is
that this behaviour is reliant on the behaviour of the python3-pep517
package. (Maybe this *was* a network-related issue in the past as
well… but this matters very little now.)

As to a solution, though, I think this is somewhat blocking on Mattia's
expertise in the generation of the Python test recommends. Are we, in
essence, trying to parse the following data in setup.py?

    install_requires=[
        "python-magic",
        "libarchive-c",
    ],
    extras_require={
        "distro_detection": ["distro"],
        "cmdline": ["argcomplete", "progressbar"],
        "comparators": [
            "androguard",
            "binwalk",
            "defusedxml",
            "guestfs",
            "jsondiff",
            "python-debian",
            "pypdf",
            "pyxattr",
            "rpm-python",
            "tlsh",
        ],
    },

If so, we don't necessarily have to wholesale move to pyproject.toml;
instead, we could rejig setup.py...?


Chris

[0] https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/325#note_366185
[1] https://salsa.debian.org/reproducible-builds/diffoscope/commit/c341b63a4c8cfe56be883b43b4e4faff71fc060e