[Pkg-utopia-maintainers] Bug#994096: /var/lib/dbus/machine-id breaks reproducible-builds

Sat Sep 11 18:04:27 BST 2021

Package: dbus
Version: 1.12.20-2
Severity: important

I am building Debian Live images using mmdebstrap.
I noticed they were not reproducible.
I eventually narrowed it down to dbus:

    bash5$ for i in 1 2; do SOURCE_DATE_EPOCH=1 mmdebstrap bullseye $i.squashfs --logfile=$i.log || echo DERP; done && cksum 1.squashfs 2.squashfs
    2581161214 57483264 1.squashfs
    2581161214 57483264 2.squashfs

    bash5$ for i in 1 2; do SOURCE_DATE_EPOCH=1 mmdebstrap bullseye $i.squashfs --logfile=$i.log --include=dbus || echo DERP; done && cksum 1.squashfs 2.squashfs
    1368751007 57929728 1.squashfs
    2280740337 57929728 2.squashfs

    bash5$ mmdebstrap --version
    mmdebstrap 0.7.5

After some investigation, I discovered that inside the image (1.squashfs), /etc/machine-id is absent, but /var/lib/dbus/machine-id is present.

An unreleased version of mmdebstrap changes sets /etc/machine-id to "uninitialized".

    https://gitlab.mister-muffin.de/josch/mmdebstrap/commit/3c37d692a0d0bbc829b0832ad20ff472c74a7a62

This did NOT fix the problem:

    bash5$ git pull
    Already up to date.

    bash5$ git describe --always --tags
    0.7.5-50-g7d472ca

    bash5$ export PATH=$PWD:$PATH

    bash5$ which mmdebstrap
    /home/twb/Desktop/mmdebstrap/mmdebstrap

    bash5$ for i in 1 2; do SOURCE_DATE_EPOCH=1 mmdebstrap bullseye /tmp/$i.squashfs --verbose --logfile=/tmp/$i.log --include=dbus || echo DERP; done && cksum /tmp/[12].squashfs
    3289132048 57929728 /tmp/1.squashfs
    449302736 57929728 /tmp/2.squashfs
    bash5$ sudo mkdir /tmp/1 /tmp/2
    bash5$ sudo mount /tmp/1.squashfs /tmp/1
    bash5$ sudo mount /tmp/2.squashfs /tmp/2
    bash5$ sudo mount -t tmpfs none /tmp/1/dev
    bash5$ sudo mount -t tmpfs none /tmp/2/dev
    bash5$ sudo GIT_PAGER= git diff --no-index /tmp/1 /tmp/2
    diff --git a/tmp/1/var/lib/dbus/machine-id b/tmp/2/var/lib/dbus/machine-id
    index 3edc593..ec187b7 100644
    --- a/tmp/1/var/lib/dbus/machine-id
    +++ b/tmp/2/var/lib/dbus/machine-id
    @@ -1 +1 @@
    -0b51d6ce222c427f9cf10a49e2db92a7
    +e003fd9d3ca240d786eda2c574f7b791

    bash5$ cat /tmp/1/etc/machine-id
    uninitializedbash5$ cat /tmp/2/etc/machine-id
    uninitializedbash5$ sudo umount /tmp/1/dev /tmp/2/dev /tmp/1 /tmp/2

I also had a guess trying to run "dbus-uuiden --ensure", but it did not help:

    bash5$ mmdebstrap bullseye /tmp/1.squashfs --include=dbus --customize-hook='grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; printf uninitalized >$1/etc/machine-id; printf uninitalized >$1/etc/machine-id $1/var/lib/dbus/machine-id; chroot $1 dbus-uuidgen --ensure; ls -lids $1/etc/machine-id $1/var/lib/dbus/machine-id; grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; false'
    I: automatically chosen mode: unshare
    I: chroot architecture amd64 is equal to the host's architecture
    I: automatically chosen format: squashfs
    I: using /tmp/mmdebstrap.CxI5RYbsZJ as tempdir
    W: tar2sqfs does not support extended attributes from the 'system' namespace
    I: running apt-get update...
    done
    I: downloading packages with apt...
    done
    I: extracting archives...
    done
    I: installing essential packages...
    done
    I: installing remaining packages inside the chroot...
    done
    done
    I: running --customize-hook in shell: sh -c 'grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; printf uninitalized >$1/etc/machine-id; printf uninitalized >$1/etc/machine-id $1/var/lib/dbus/machine-id; chroot $1 dbus-uuidgen --ensure; ls -lids $1/etc/machine-id $1/var/lib/dbus/machine-id; grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; false' exec /tmp/mmdebstrap.CxI5RYbsZJ
    /tmp/mmdebstrap.CxI5RYbsZJ/etc/machine-id:cb3779e3f4e64aebab438a1dee355b59
    /tmp/mmdebstrap.CxI5RYbsZJ/var/lib/dbus/machine-id:cb3779e3f4e64aebab438a1dee355b59
    4112969 4 -r--r--r-- 1 root root 12 2021-09-12 02:39 /tmp/mmdebstrap.CxI5RYbsZJ/etc/machine-id
    4117680 4 -rw-r--r-- 1 root root 33 2021-09-12 02:39 /tmp/mmdebstrap.CxI5RYbsZJ/var/lib/dbus/machine-id
    /tmp/mmdebstrap.CxI5RYbsZJ/etc/machine-id:uninitalized
    /tmp/mmdebstrap.CxI5RYbsZJ/var/lib/dbus/machine-id:cb3779e3f4e64aebab438a1dee355b59
    E: run_chroot failed: E: command failed: grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; printf uninitalized >$1/etc/machine-id; printf uninitalized >$1/etc/machine-id $1/var/lib/dbus/machine-id; chroot $1 dbus-uuidgen --ensure; ls -lids $1/etc/machine-id $1/var/lib/dbus/machine-id; grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; false
    W: listening on child socket failed: 
    reading tar header: short read
    W: creating tarball failed: E: failed to run tar2sqfs --quiet --no-skip --force --exportable --compressor xz --block-size 1048576 /tmp/1.squashfs

    I: removing tempdir /tmp/mmdebstrap.CxI5RYbsZJ...
    bash5$ cat /etc/machine-id
    029d2e3fb4ee4d5eaa67c315db3ba66d

Let's just check this issue affects sid (latest dbus in Debian)...

    bash5$ mmdebstrap sid /tmp/1.squashfs --include=dbus --customize-hook='grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; printf uninitalized >$1/etc/machine-id; printf uninitalized >$1/etc/machine-id $1/var/lib/dbus/machine-id; chroot $1 dbus-uuidgen --ensure; ls -lids $1/etc/machine-id $1/var/lib/dbus/machine-id; grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; false'
    I: automatically chosen mode: unshare
    I: chroot architecture amd64 is equal to the host's architecture
    I: automatically chosen format: squashfs
    I: using /tmp/mmdebstrap.ZLY1AQ7Gjd as tempdir
    W: tar2sqfs does not support extended attributes from the 'system' namespace
    I: running apt-get update...
    done
    I: downloading packages with apt...
    done
    I: extracting archives...
    done
    I: installing essential packages...
    done
    I: installing remaining packages inside the chroot...
    done
    done
    I: running --customize-hook in shell: sh -c 'grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; printf uninitalized >$1/etc/machine-id; printf uninitalized >$1/etc/machine-id $1/var/lib/dbus/machine-id; chroot $1 dbus-uuidgen --ensure; ls -lids $1/etc/machine-id $1/var/lib/dbus/machine-id; grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; false' exec /tmp/mmdebstrap.ZLY1AQ7Gjd
    /tmp/mmdebstrap.ZLY1AQ7Gjd/etc/machine-id:9a85960cbe9a4be78f6359211a35cddb
    /tmp/mmdebstrap.ZLY1AQ7Gjd/var/lib/dbus/machine-id:9a85960cbe9a4be78f6359211a35cddb
    4128767 4 -r--r--r-- 1 root root 12 2021-09-12 02:44 /tmp/mmdebstrap.ZLY1AQ7Gjd/etc/machine-id
    4133499 4 -rw-r--r-- 1 root root 33 2021-09-12 02:44 /tmp/mmdebstrap.ZLY1AQ7Gjd/var/lib/dbus/machine-id
    /tmp/mmdebstrap.ZLY1AQ7Gjd/etc/machine-id:uninitalized
    /tmp/mmdebstrap.ZLY1AQ7Gjd/var/lib/dbus/machine-id:9a85960cbe9a4be78f6359211a35cddb
    E: run_chroot failed: E: command failed: grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; printf uninitalized >$1/etc/machine-id; printf uninitalized >$1/etc/machine-id $1/var/lib/dbus/machine-id; chroot $1 dbus-uuidgen --ensure; ls -lids $1/etc/machine-id $1/var/lib/dbus/machine-id; grep ^ $1/etc/machine-id $1/var/lib/dbus/machine-id; false
    W: listening on child socket failed: 
    reading tar header: short read
    W: creating tarball failed: E: failed to run tar2sqfs --quiet --no-skip --force --exportable --compressor xz --block-size 1048576 /tmp/1.squashfs

    I: removing tempdir /tmp/mmdebstrap.ZLY1AQ7Gjd...

I am not sure how to fix this.
The references to machine-id in the dbus sources confused me.
It seems like sometimes it's a link, sometimes it's a symlink, sometimes it's a copy.

AFAICT there's no mention of dbus's machine-id supporting "uninitialized".

-- System Information:
Debian Release: 11.0
  APT prefers stable-security
  APT policy: (990, 'stable-security'), (990, 'stable'), (500, 'proposed-updates'), (500, 'unstable'), (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 5.10.0-8-amd64 (SMP w/8 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages dbus is related to:
ii  dbus-x11      1.12.20-2
ii  systemd       247.3-6
ii  systemd-sysv  247.3-6