Bug#1019742: reprotest: add a variation that sets DEB_BUILD_OPTIONS=nocheck

[Vagrant Cascadian]

On 2022-09-14, Philip Hands wrote:
> Vagrant Cascadian <vagrant at reproducible-builds.org> writes:
>
>>> but also
>>> (given that the tests will have passed during the normal build) the tests
>>> failing during the varied build seems unlikely to be identifying faults that are
>>> worth fixing, and so is just a waste of cycles.
>>
>> How do you know weather the bugs it is identifying are worth fixing? It
>> could also identify non-deterministic failures, or failures triggered by
>> specific build environment configurations...
>
> The point is that if the package is reproducible, then the fact that its
> tests fail when run in a weird environment (that may never be found in
> the wild) seems rather likely to be finding errors in the tests rather
> than errors in the program that gets shipped.

Fair point! 

> Of course, if the package is not reproducible, the tests may well fail
> because the package ends up containing new bugs that are only present in
> the variant-built package, but then its also going to show up as
> non-reproducible, so does that really make a difference?

True, though it may make things harder to verify reproducibility in
practice, especially if it is a fairly "normal" variation that triggers
the issue...


It is a balancing act...

I guess I'd be fine with the defaults to go either way, but it would be
important to be able to enable or disable however this gets implemented.


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20220914/53001224/attachment.sig>