source-only builds and .buildinfo

[Ian Jackson]

(Resending with the right CC for reproducible-builds at lists.a.d.o)

Hi.  I'm widening the scope of this thread because I think the
reproducible builds folks might have an opinion.  (Holger said on IRC
that they'd welcome a CC.)  So, I'm going to recap.


dpkg-buildpackage -S (which is the conventional way to build a
source-only upload) generates a .buildinfo file, which ends up
appearing in the .changes.  (I don't know what dak does with it.)

Sean tripped over this in the context of developing a new dgit
operation mode `dgit push-source'.  I found the generation of a
.buildinfo questionable so we asked debian-dpkg.  Guillem pointed me
to this FAQ entry:

  https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Why_are_.buildinfo_files_always_generated_with_dpkg-buildpackage.3F 

As I wrote to Guillem, quoting the FAQ:

>   By default dpkg-buildpackage does active tasks such as cleaning via
>   debian/rules, and makes sure that the dependencies from Build-Depends
>   are satisfied as these are needed by the clean target. In addition the
>   clean target can perform any kind of action that will affect the
>   source package, which is also part of the build, and should be by
>   default also reproducible
> 
> I think what you mean here is that one might have a source package
> which is not a fixed point under `debian/rules clean' for all
> reasonable combinations of the build-deps.  I think this is a buggy
> package but in practice it seems that many packages are buggy in this
> way.
> 
> Indeed IMO it is a defect of our overall design that it the concept of
> a `non-reproducible source package' even exists.  Sources are the
> input to builds, not an output, so the question of reproducing them
> does not arise.  That our system as a whole can sometimes mutate the
> source package all by itself is a bug.
> 
> However, these are not considerations for dgit in this context, since
> what dgit uploads is always guaranteed to be equal to the input.
> Often the user will have dgit use `git clean' rather than rules clean;
> and even if they don't, dgit will check that the results were the
> same.
> 
> That is, even with the .buildinfo, someone who gets the .dsc cannot
> know whether the rules clean target is correct (or to put it another
> way, under what conditions the source tree is a fixed point under
> rules clean), because dgit has not necessarily run rules clean at all.
> I'm sure there are other vcsish tools which have the same property.
> 
> (Also, and I hesitate to make this argument because of course I
> support reproducible builds, but: if the .buildinfo is not useful,
> then it's an unwarranted privacy violation.)
> 
> So I think for `dgit push-source', there should be no .buildinfo ?
> At least, unless dgit ran the clean target.
> 
> This suggests to me that dgit push-source should use dpkg-source
> rather than dpkg-buildpackage, as you note in later in the FAQ entry:
> 
>   If the intention is to just produce a source package instead of an
>   actual build to upload, then using dpkg-source is always the better
>   option.
> 
> This wording is a bit unclear.  It conflates `build' and `for upload'.
> I think regarding `dgit push-source' as a build is perverse.
> 
> dgit would have to run dpkg-genchanges.
> 
> Alternatively dgit could strip out the .buildinfo, depending on
> whether it ran rules clean.

What do you think ?

(The background here is that `dgit push-source' wants to verify for
itself that the .changes file it is uploading is really source-only.
Because of the possible presence of extraneous (eg BY-HAND) build
artefacts in .changes, Guillem suggested comparing the .changes to the
.dsc.  But of course the .changes contains not only the .dsc and the
files named in it, but also the .buildinfo.)

Thanks,
Ian.