Debian unstable (and more) rep-bui problems (Was: Re: Rant about Debian reproducibility environment)

[Steffen Nurpmeso]

Arnout Engelen wrote in <CACDuYhu4+AHtkPrqHQqQOksvWGgbtLv+Voz98zrOB3LffS\
nC_g at mail.gmail.com>:
 |>|  outcome on two different build hosts, unless the actual build
 |>|  environment is the same to the detail.
 |
 |The goal of Reproducible Builds is to increase our confidence that
 |no malware was injected into the artifact during the build process.
 |
 |To achieve this, we build from source on different
 |machines and check whether the result is the same.
 |
 |There is a tension between 2 concerns here: on the one hand, you
 |want those machines to be as diverse as possible: the larger
 |the difference between the machines, the higher your confidence in
 |the absence of foul play (since an attacker would have to find a way
 |to impact all those variations in machines).
 |On the other hand, of course it is unreasonable to expect the same
 |results on machines that are too wildly different.
 |
 |Which differences a build procedure should be resistant against
 |to be considered 'reproducible' is not always clear-cut. I understand
 |this can be frustrating when apparently previously the Debian
 |rebuilders didn't exercise a certain difference while now they do.

I can understand that and absolutely agree.

 |On the other hand, just the fact that this was constant before
 |doesn't immediately mean this is a defect in the rebuilding
 |infrastructure, either.
 |
 |So the question is whether it is reasonable to require all builders
 |to have identical MAKEFLAGS. There are definitely things in the
 |MAKEFLAGS that I wouldn't expect to influence the resulting
 |artefact, such as the build parallelism. On the other hand,
 |of course it is entirely reasonable for application-level configuration
 |flags to change the output, and making those appear in the output
 |of something like "mailx -v -Xversion -Xx" sounds legitimate to
 |me as well.

Thank you.

 |Would there be any way keep only the 'application-level' options
 |from the MAKEFLAGS but leave the 'build-level' options out?
 |(you mention 'test1/ and test2/ or so', but I'm not sure what
 |exactly what's going on there).

Starting with the next release (hopefully next week) we will split
MAKEFLAGS and only bake in anything after the -- command line
option terminator.  (This likely is GNU make only, Solaris would
go for distributed make, maybe, but is hopefully sufficient for
our purpose here.)  Like this configuration options should remain,
and make(1) specific flags like job control should be kept
outside.

On Debian unstable the differences originate in the use of
-ffile-prefix-map=/build/1st/ against =/build/2/, i admit i did
not know this compiler option.  I know now, cool, but, hmmm.
For ArchLinux the difference was -j32 against -j31, which should
be fixed with the next release i hope.

Thanks for an Engelen on a Friday the 13th, a nice weekend and
Ciao from Germany,

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)