Comparing bullseyes to bullseyes

[Vagrant Cascadian]

So, we have some reasonable nubmers about actual reproducibility for
Debian bullseye from the "beta" tests; looks to be over 90%
reproducible, which is great considering it is compared against packages
people actually use in the real world!

The "beta" tests wil miss out on toolchain fixes introduced since the
last time the package was built, as it builds using the same toolchain,
whereas the other tests use the toolchains currently in bullseye.

  https://tests.reproducible-builds.org/debian/bullseye/index_suite_amd64_stats.html
  ~96% reproducible
  29647+828+334+23 = 30832 packages tested (excluding 64 non-amd64 and non-arch-all)

  https://beta.tests.reproducible-builds.org/debian/results/bullseye_full.amd64+all.png
  ~92% reproducible
  26403+1398+62+820 = 28683 packages tested

I did notice an oddity in that the tests have ~1.8k more packages being
tested than the "beta" tests.  Not sure how that ~1.8k packages would
skew the results, but even then, the numbers still probably look pretty
good.

I know the results from the "beta" tests do separate arch:amd64 builds
and arch:all builds and then combines the data somehow, where the other
tests do arch:amd64+arch:all builds in a single pass per source package;
could that be related to the difference in total packages tested?

Holger also mentioned there were 500+ packages in bullseye without
.buildinfo files; I presume "beta" can only test packages with a
.buildinfo file, but does it log "missing .buildinfo" in some way?

Any other thoughts how the two different systems might be counting
packages differently?


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20211026/17f0159a/attachment.sig>