[Reproducible-builds] Heads up: Upcoming dpkg-buildpackage -j precedence change

[Guillem Jover]

[ reproducible-builds people, please see below. ]

Hi!

On Tue, 2015-05-12 at 10:02:27 +0100, Jonathan Dowland wrote:
> On Mon, May 11, 2015 at 08:40:16PM +0200, Guillem Jover wrote:
> > $ make -jN -f debian/rules build
> > 
> > and
> > 
> > $ DEB_BUILD_OPTIONS=parallel=N debian/rules build
> 
> I prefer the latter behaviour but the former brevity. Would you consider
> something like -J in dpkg-buildpackage adjusting parallel= in
> DEB_BUILD_OPTIONS, and perhaps in the future phasing out -j?

Sure, adding either an option to disable the forced parallel runs in
combination with -j, or a new option such as -J seems fine to me.

The “nice” thing about -j is that it allows anyone to naturally exercise
parallel builds (like they'd do with an upstream project) and possibly
notice if something does not support them. Getting rid of it would mean
that the current marking of packages might be even slower than it is.
Because we have to opt-in for every and each package in the archive,
with the usual multi-year Debian adoption rates and probably longer in
this case. So I think having both semantics at hand is useful.

And of course «dpkg-buildpackage -jN» does not cover build systems that
do not use make, but I've always thought of DEB_BUILD_OPTIONS=paralle=N
as a way to request that.

> > Actually Makefiles that do not support parallel builds and do not
> > declare so with .NOTPARALLEL: are buggy, because upstreams do not
> > have any standardized opt-in way to honor a parallel build request.
> 
> Now that rules files have to be makefiles; I wonder if this should be clarified
> in policy, one way or the other. That is: should debian/rules be assumed to
> support parallel building by default, or not? I imagine the saner default for
> us would be not (even though I am a big fan of parallel builds).

IMO dpkg-buildpackage should assume yes, in the same way it assumes
packages are cross-buildable, and we don't go around marking them as
such. But I guess for Debian that depends on how much of our packaging
and upstream build systems are parallel buildable. I'd have assumed that
with projects like Gentoo around and multicore systems being so common
nowadays, many things would be parallel buildable already, although I
don't have numbers…

… but, we do have a way to check if a package is parallel buildable now,
through reproducible builds. If a package outputs exactly the same with
dpkg-buildpackage -j1 and -jauto (w/o DEB_BUILD_OPTIONS) then it means
the package is good, otherwise it might need fixing one way or another.

Thanks,
Guillem