testing migration aging and reproducible builds

[Vagrant Cascadian]

On 2019-07-21, Chris Lamb wrote:
> So, the devil is very much in the details here, alas. Whilst I am a
> definite +1 on this idea the day-to-day experience may not be ideal
> so your thoughts are very welcome.
>
> Just to take one recent example: I noticed yesterday a regression
> whereby one of our tools (strip-nondeterminism) is causing a fair
> number of Java packages to be unreproducible — it would seem a little
> antisocial to block them from migrating to testing when it was "our"
> fault. It is not really within the power or remit of the maintainers
> in-question to fix this particular issue, and we should not implicitly
> encourage maintainers to manually (!) fix it in each of the affected
> packages just to get it to migrate...

Indeed.


> Note that delaying migration here has quite a different consent
> and social dynamic to autopkgtest failures as the maintainers have,
> by uploading a package that contains autopkgtests, implicitly opted
> into the committment to ensure they continue to pass.
>
> Anyway, your thoughts on this important angle?

This makes me think to decrease the delay for reproducible packages
rather than increase the delay for unreproducible ones? Though then
you'd want it to be a small increment...

It's more of an incentive that way than a punishment, and humans tend to
be more motivated by incentives rather than punishment...

Still would hate for maintainers to be overly zealous in on-off fixes.


>> For this to be possible, we would need to have an automated way to get 
>> data about the source packages
>
> Regarding the technical side of the implementation, we currently
> generate this (~3MB bzipped) JSON file:
>
>   https://tests.reproducible-builds.org/debian/reproducible.json.bz2
>
> ... that (unless I am mistaken) is the data being used by qa.debian.org.

I think it's:

  https://tests.reproducible-builds.org/reproducible-tracker.json

Which only shows results for "bullseye" (formerly only "buster).

We wanted to filter out the unstable and experimental suites from the
tracker which may have larger numbers of variations (e.g. build path),
some of which may be just be distracting to developers at this point
until we have a more complete fix for those issues.

I'm not sure relying on our test infrastructure at the moment is the
right approach long-term, maybe fine in the short-term.

We really need to start doing verification of buildd builds rather than
simply rebuilding packages twice with variations. Was hoping to put
together a BoF this week with DSA about that; maybe release team would
also be interested?


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20190721/d6957d79/attachment.sig>