bit by bit identical chroot creation (was Re: Debian and our frenemies of containers and userland repos)

[Johannes Schauer]

Hi,

Quoting Holger Levsen (2019-10-08 09:45:08)
> On Mon, Oct 07, 2019 at 01:43:18PM +0200, Johannes Schauer wrote:
> [...]
> > Downloading "random binary from the internet" is less of a problem if we can
> > create images which are bit-by-bit identical to checksums that we can verify
> > through a trusted service. This is also already provided by mmdebstrap:
> > 
> >     $ SOURCE_DATE_EPOCH=1570448177 mmdebstrap --variant=essential unstable - | sha256sum
> >     [...]
> >     f40a3d2e9e168c3ec6270de1be79c522ce9f2381021c25072353bb3b5e1703d6  -
> >     $ SOURCE_DATE_EPOCH=1570448177 mmdebstrap --variant=essential unstable - | sha256sum
> >     [...]
> >     f40a3d2e9e168c3ec6270de1be79c522ce9f2381021c25072353bb3b5e1703d6  -
> 
> wow, neato, I wasn't aware of this. very cool!
> 
> I don't think debootstrap does this already, or does it?
>
> And, does this work for mmdebstrap'ing buster too? (whether using
> mmdebstrap from unstable or buster...)

lets find out!

    $ sudo mmdebstrap --include=mmdebstrap,debootstrap,diffutils buster ./debian-buster
    [...]
    $ sudo chroot ./debian-buster
    # cat /etc/apt/sources.list
    deb http://deb.debian.org/debian buster main
    deb http://deb.debian.org/debian buster-updates main
    deb http://security.debian.org/debian-security buster/updates main
    # SOURCE_DATE_EPOCH=1570522957 mmdebstrap --variant=minbase unstable - | sha256sum
    [...]
    e43ab25109a1f9e73fcb9de698912e25d7402c2aef4445a46719621b517901bf  -
    # SOURCE_DATE_EPOCH=1570522957 mmdebstrap --variant=minbase unstable - | sha256sum
    [...]
    e43ab25109a1f9e73fcb9de698912e25d7402c2aef4445a46719621b517901bf  -
    # SOURCE_DATE_EPOCH=1570522957 mmdebstrap --variant=minbase buster - | sha256sum
    [...]
    a1f4bc1f1c8e4a8942a1cbeed61f02556533d0381de0f9befe618246fec08af7  -
    # SOURCE_DATE_EPOCH=1570522957 mmdebstrap --variant=minbase buster - | sha256sum
    [...]
    a1f4bc1f1c8e4a8942a1cbeed61f02556533d0381de0f9befe618246fec08af7  -
    # SOURCE_DATE_EPOCH=1570522957 debootstrap --variant=minbase unstable ./debian-unstable-A
    [...]
    # SOURCE_DATE_EPOCH=1570522957 debootstrap --variant=minbase unstable ./debian-unstable-B
    [...]
    # diff -rq ./debian-unstable-A ./debian-unstable-B
    Files debian-unstable-A/var/cache/ldconfig/aux-cache and debian-unstable-B/var/cache/ldconfig/aux-cache differ
    Files debian-unstable-A/var/log/alternatives.log and debian-unstable-B/var/log/alternatives.log differ
    Files debian-unstable-A/var/log/bootstrap.log and debian-unstable-B/var/log/bootstrap.log differ
    Files debian-unstable-A/var/log/dpkg.log and debian-unstable-B/var/log/dpkg.log differ

Since it is not crucial to have these files in a chroot after creating it (they
will all be re-created) mmdebstrap just removes them. Obviously, mmdebstrap
cannot do much about reproducibility coming from many other sources like
database creation in maintainer scripts or issues like these:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=917386
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=917407

Thanks!

cheers, josch
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: signature
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20191008/8cb1ccb6/attachment.sig>