[Reproducible-builds] Bug#138409: Bug#138409: dpkg-dev: please add support for .buildinfo files

[Guillem Jover]

On Sat, 2016-01-30 at 15:18:30 +0100, Jérémy Bobbio wrote:
> Guillem Jover:
> > Lunar:
> > > I think the proposed patch is missing a field to record some environment
> > > variables that can affect the build process. Right now, I'm thinking of
> > > DEB_BUILD_OPTIONS and DEB_flag_{SET,STRIP,APPEND,PREPEND} (from
> > > dpkg-buildflags). Maybe others? Build profiles?
> > >
> > > Initially I was against recording such information, but now that we
> > > understand the purpose of .buildinfo files better and not mandate that
> > > they be reproducible themselves, it doesn't matter if one contains
> > > `DEB_BUILD_OPTIONS=parallel=4` and the other
> > > `DEB_BUILD_OPTIONS=parallel=16`. It is the responsibility of users
> > > trying to recreate a given package to filter this out.
> > 
> > Hmm right, makes sense, but I see this also to be a bit problematic.
> > There are many variables that do affect the build which we'd need to
> > record. Including all of the them seems like another privacy
> > concerning issue. Whitelisting, we might end up missing some but it's
> > privacy-safe; blacklisting we might end-up including sensitive ones,
> > but not miss any build-related ones, which is privacy-unsafe.
> > 
> > Some things that come to mind that do affect the build in a significant
> > way: CC, LD_LIBRARY_PATH, DEB*, DPKG_*, PATH, MAKEFLAGS.
> > 
> > The traditional build flags (i.e. CFLAGS, LDFLAGS, etc) might also affect
> > the build depending on the rules file.
> 
> I was thinking of a whitelist approach and to start with use cases we
> can already think of, adding more variables to record if we identify
> them as missing in the future.

That works for me, and it's no worse than our current support for
build flags, where we have a whitelist too. So with new build flags
we need to explicitly add support for them too. Althought here we'd
like to catch build variables even if dpkg-buildflags does not support
them (say CXX).

> How about naming the field “Environment-Variables”?

Hmm, or Environment, or Build-Environment, which reminds me that I've
found the usage of Build-Environment (as the list of transitively
required packages) slightly confusing, precisely because the first
thing that comes to mind with environment is the variable space.

Perhaps we should consider renaming that one? Say Build-Packages (but
that might be confusing), Build-Depends-Used, or something else? We
also already have a Built-Using field too (although for source
packages not binary ones, with a name I've also found slightly
confusing as being too generic).

> I will also add another vendor hook for the list of variables.

I don't think this is necessary in this case, all current variables
seem pretty dpkg-generic to me. Or do you see any that is
Debian-specific?

> > Build profiles are already recorded in the binary packages, but having
> > that in the .buildinfo file seems right as it makes it easier to
> > reproduce the build environment.
> 
> Should it be a new field or would recording the DEB_BUILD_PROFILES
> variable be enough?

These can be passed also through the -P option to dpkg-buildpackage and
dpkg-checkbuilddeps, but this is not currently possible for all scripts
that use build profiles, and that's why dpkg-buildpackage sets the
DEB_BUILD_PROFILES variable from the command-line option value, so I
think just recording the environment is enough for now. We can always
also record them in a Build-Profiles field if need be.

Thanks,
Guillem