[Reproducible-builds] From srebuild sbuild-wrapper to debrebuild

Johannes Schauer josch at debian.org
Tue Aug 2 20:49:00 UTC 2016


Hi,

On Mon, 09 May 2016 21:07:40 +0200 Johannes Schauer <josch at debian.org> wrote:
> The main disadvantage of the current srebuild implementation is, that it will
> only make use of a single snapshot.d.o timestamp. This makes it impossible to
> reproduce situations where packages are not built in a clean chroot, in a
> partially updated chroot or in a chroot mixing different suites. To assemble
> a chroot with the right package versions, sbuild could retrieve the exact
> right debs from snapshot.d.o.

I was thinking about this issue again and thought that instead of creating a
wrapper for sbuild which then uses a chroot-setup hook to install the
dependencies, what I should instead do is to let sbuild itself accept
.buildinfo files and then do the right thing like:

 - use snapshot.d.o to retrieve the right timestamps needed to gather all
   packages
 - mangle the build dependencies such that the source package now depends on
   the exact right package versions and let the resolver figure out the rest
   (thanks Benjamin for that idea)
 - check whether the generated binaries produce the same checksum as given in
   the supplied buildinfo file

But then on IRC, HW42 suggested to approach this problem differently. Instead
of integrating the functionality of figuring out the right repositories to
reproduce the contents of a buildinfo file into sbuild, write a tool that can
drive any package builder (like pbuilder).

I now wrote such a script. It currently supports sbuild or manual installation
by showing the correct sources.list. For both it prints the correct command
line invocations. Advantages over the old sbuild hook-based script attached to
initial post are:

 - package versions can come from multiple snapshot timestamps
 - theoretically works for more package builders than just sbuild (pbuilder
   support is missing because I don't know enough about pbuilder)
 - uses Dpkg::Checksums to parse and verify the hash and size fields instead of
   doing it manually
 - uses apt to download and manage Packages files instead of doing it manually
 - allows to add additional repositories like reproducible.alioth.debian.org
   (which is hardcoded so far)
 - uses base-files and dpkg version to estimate a base Debian release
 - drastically reduce number snapshot.d.o API queries by only querying for
   missing packages

Limitations:

 - There is no nice command line interface with options and switches yet
 - You cannot yet supply additional initial archives
   (reproducible.alioth.debian.org is hardcoded)
 - It only considers Debian main
 - It only considers official Debian (and not ports)
 - It only considers Debian unstable from snapshot.d.o
 - You have to manually run sbuild/pbuilder with the displayed command and then
   manually verify if the .buildinfo file stayed the same

What is still needed:

 - a good name (I named it debrebuild for now because it is Debian centric and
   rebuilds a package that was built before to check if the checksums can be
   reproduced locally. This is the main difference to reprotest which does not
   require an existing build but checks for reproducibility by building the
   same software twice in different environments)
 - a nice home for the script to live
 - somebody maintaining the software and making it more user friendly by adding
   a nice command line interface and writing a README file and/or man page
 - maybe let the script execute the sbuild/pbuilder command it suggests to run
   as well. This would allow the script to check the output for plausibility.

Usage:

   debrebuild.pl package.buildinfo

Depends:

   apt-get install --no-install-recommends libdpkg-perl libwww-perl libdatetime-format-strptime-perl

Have fun!

cheers, josch
-------------- next part --------------
A non-text attachment was scrubbed...
Name: debrebuild.pl
Type: text/x-perl
Size: 14684 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/reproducible-builds/attachments/20160802/e5d422f8/attachment.pl>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: signature
URL: <http://lists.alioth.debian.org/pipermail/reproducible-builds/attachments/20160802/e5d422f8/attachment.sig>


More information about the Reproducible-builds mailing list