[pkg-lxc-devel] Bug#1078445: autopkgtest: debian/tests/lxc-old-testbed no longer works: cannot find Ubuntu 14.04 signing key

[Simon McVittie]

Source: autopkgtest
Version: 5.38
Severity: normal
User: autopkgtest at packages.debian.org
Usertags: lxc
X-Debbugs-Cc: lxc-templates at packages.debian.org, ubuntu-keyring at packages.debian.org, dimitri.ledkov at canonical.com, debian-lts at lists.debian.org

In my autopkgtest-virt-qemu test environment (in a sid VM),
debian/tests/lxc-old-testbed can no longer create an Ubuntu 14.04 'trusty'
container and is therefore skipped:

> autopkgtest [15:34:53]: test lxc-old-testbed: [-----------------------
> Using http://192.168.122.1:3142 as container proxy
> lxc-create: autopkgtest-trusty-amd64: ../src/lxc/storage/btrfs.c: btrfs_create: 938 Inappropriate ioctl for device - Failed to create btrfs subvolume "/var/lib/lxc/autopkgtest-trusty-amd64/rootfs"
> lxc-create: autopkgtest-trusty-amd64: ../src/lxc/storage/zfs.c: zfs_create: 742 Failed to create zfs dataset "zfs:lxc/autopkgtest-trusty-amd64": lxc-create: autopkgtest-trusty-amd64:
> lxc-create: autopkgtest-trusty-amd64: ../src/lxc/storage/lvm.c: do_lvm_create: 165 Failed to create logical volume "autopkgtest-trusty-amd64": lxc-create: autopkgtest-trusty-amd64: ../src/lxc/utils.c: run_command_internal: 1813 Failed to exec command
> lxc-create: autopkgtest-trusty-amd64: ../src/lxc/storage/lvm.c: lvm_create: 623 Error creating new logical volume "lvm:/dev/lxc/autopkgtest-trusty-amd64" of size "1073741824 bytes"
> Checking cache download in /var/cache/lxc/trusty/rootfs-amd64 ...
> Installing packages in template: eatmydata,language-pack-en
> Downloading ubuntu trusty minimal ...
> W: Cannot check Release signature; keyring file not available /usr/share/keyrings/ubuntu-archive-removed-keys.gpg
> I: Retrieving InRelease
> I: Retrieving Release
> lxc-create: autopkgtest-trusty-amd64: E: Failed getting release file http://archive.ubuntu.com/ubuntu/dists/trusty/Release
> SKIP: Unable to build autopkgtest-trusty-amd64 container
> ../src/lxc/lxccontainer.c: create_run_template: 1589 Failed to create container from template
> lxc-create: autopkgtest-trusty-amd64: ../src/lxc/tools/lxc_create.c: lxc_create_main: 318 Failed to create container autopkgtest-trusty-amd64
> autopkgtest [15:34:54]: test lxc-old-testbed: -----------------------]

I believe this is because lxc-templates uses
/usr/share/keyrings/ubuntu-archive-removed-keys.gpg to get the
archive-signing key that was current at the time that Ubuntu 14.04 was
fully-supported, but ubuntu-keyring no longer ships that file since
2023.11.28.1-0.1.

As context for the cc'd maintainers, the purpose of this test is to make
sure that autopkgtest's ability to operate on extremely old testbeds
has not regressed. For the benefit of the Debian (E)LTS and Ubuntu ESM
teams, autopkgtest aims to be able to test Debian 8 'jessie' or newer and
Ubuntu 14.04 'trusty' or newer, but if we can no longer easily verify
this programmatically at release time, then that functionality should
be considered "at risk".

One way to solve this would be for ubuntu-keyring to provide
/usr/share/keyrings/ubuntu-archive-removed-keys.gpg again, analogous to
the way debian-archive-keyring provides a keyring of deprecated keys
(it doesn't need to be trusted, it just needs to exist); or for some
other appropriate package to provide a key that was used to sign trusty,
and for lxc-templates to use that key automatically when bootstrapping
a trusty container.

Another way to solve this would be to switch the lxc-old-testbed test to
construct a Debian 8 'jessie' testbed (presumably from archive.debian.org,
and possibly setting it to use an older archive-signing key).

If it would be easier to do this testing under an autopkgtest backend
that is not lxc (ideally podman or unshare) then that would also be fine.

Or, if the Debian (E)LTS and Ubuntu ESM teams are no longer interested in
jessie and trusty, or are willing to switch to using older versions of
autopkgtest to test those older suites, then we could bump up the minimum
system-under-test that we aim to support, which would also allow removing
various pieces of technical debt and legacy code from autopkgtest.

    smcv