source-only builds and .buildinfo

Wed Jun 21 14:08:37 UTC 2017

On 2017-06-21, Ian Jackson wrote:
> Daniel Kahn Gillmor writes ("Re: source-only builds and .buildinfo"):
>> On Tue 2017-06-20 18:10:49 +0100, Ian Jackson wrote:
>> > A .buildinfo file is not useful for a source-only upload which is
>> > veried to be identical to the intended source as present in the
>> > uploader's version control (eg, by the use of dgit).
>> >
>> > Therefore, dgit should not include .buildinfos in source-only uploads
>> > it performs.  If dgit sees that a lower-layer tool like
>> > dpkg-buildpackage provided a .buildinfo for a source-only upload, dgit
>> > should strip it out of .changes.
>> 
>> I often do source-only uploads which include the .buildinfo.
>> 
>> I do source-only uploads because i don't want the binaries built on my
>> own personal infrastructure to reach the public.  But i want to upload
>> the .buildinfo because i want to provide a corroboration of what i
>> *expect* the buildds to produce.
>
> This is an interesting use case which dgit should support.

Agreed!

> But I think this is not what dgit push-source should do.  Sean's
> proposed dgit push-source does not do any kind of binary package
> build.  I think this is correct.  But this means there are no binaries
> and nothing for the .buildinfo to talk about.

Yes, this makes sense for the most part.

> Do the "source-only uploads" that you are talking about mention the
> hashes of these locally-built .debs in their .buildinfo, then ?

That's the goal, sure.

I've done this with all my recent source-only uploads, and then gone
back and verified that the buildd machines produced (in most cases), the
same hashes for the .deb files.

For example, this references the buildinfo of simple-cdd 0.6.5 I
uploaded with a source-only changes file in:

  https://buildinfo.debian.net/30f7000b0025b570c7ae2202fc6fd79e4ca27798/simple-cdd_0.6.5_all

And this is a buildinfo produced over a month later on the reproducible
builds build network, on a different architecture (i386), with a
different build environment, that produced the same hashes:

  https://buildinfo.debian.net/1d300b71445ac7d756e93546a7e6b36d3c1882c7/simple-cdd_0.6.5_all

And you can check the .buildinfo in the build logs on the buildd
produced the same sha1 hashes:

  https://buildd.debian.org/status/fetch.php?pkg=simple-cdd&arch=all&ver=0.6.5&stamp=1494884527&raw=0

And then you can compare the hashes of simple-cdd packages in the
archive are the same hashes listed.

Given that at least three machines, of differing architecture, with over
a month between the packages in the build toolchain, produced the same
binary packages... I have *some* confidence that this package is
reproducible.

It's not the most complicated package, but it demonstrates that it is
now possible, for a reasonable portion of the archive, to at least
manually verify many of the builds. Some of this could be automated...

> Certainly `dgit push' will not do anything to any .buildinfo you may
> have.  I think maybe that your use case should be supported by having
> a version of dgit push which drops the .debs from the .changes, but
> leaves the .buildinfo ?  Is that how you construct these uploads now ?

I use sbuild's --source-only-changes option, which creates two .changes
files, one with the debs (ARCH.changes), and one
without(source.changes). In both cases, the .buildinfo referenced in
.changes includes hashes of the .deb files.

> (Also: is there anything right now that verifies your assertions about
> the .debs?  Not that the lack of such a thing would make the
> .buildinfos useless, but my experience is that without closing that
> loop it is likely that the arrangements for generating the .buildinfo
> are wrong somehow in a way we haven't spotted.)

There's nothing corroborating the results of .deb files in the archive
against tests.reproducible-builds.org build results, but that does
rebuild all packages in the archive with permutations of the build
environment, and logs when they aren't reproducible.

The archive is keeping the .buildinfo files uploaded with packages,
though they aren't, to my knowledge, exposed yet. But it would allow for
retroactive verification of said packages once the .buildinfo files are
available. A few relevent bugs on ftp.debian.org regarding this:

  https://bugs.debian.org/763822
  https://bugs.debian.org/862073
  https://bugs.debian.org/862538
  https://bugs.debian.org/863470

live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/reproducible-builds/attachments/20170621/529d5ab7/attachment.sig>