Build with DEB_BUILD_OPTIONS="nocheck" when testing for reproducibility?

[Vagrant Cascadian]

On 2025-03-22, Otto Kekäläinen wrote:
> I noticed https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/mariadb.html
> is failing on a single test failing in the test suite that runs after
> the actual build. While the test should of course be made more robust,
> I am not sure running those tests are relevant for testing
> reproducibility as the tests don't affect the artifacts, so it would
> actually make sense to skip those tests when testing for
> reproducibility.

Well, we have seen all too many packages where the tests do actually
change the resulting artifacts, unfortunately.

Off of the top of my head, binutils and gcc intentionally embed the
results of test suites into a .deb ... and some unintentionally embed
test suite results. This only lists a few that we have found where it is
pretty obvious:

  https://tests.reproducible-builds.org/debian/issues/unstable/test_suite_logs_issue.html

Some change based on the build in more subtle ways...

Though, now that we have https://reproduce.debian.net building against
the archive (and thus running the tests)...

Maybe it would be worth adding DEB_BUILD_OPTIONS=nocheck on the second
build for https://tests.reproducible-builds.org/debian ? It might save
some build time and find some issues that would probably be worth
fixing...


> Is there some general technique to run specific packages with
> DEB_BUILD_OPTIONS="nocheck" in reproducible builds?

Well, you can use salsa-ci reprotest job or run reprotest locally with
DEB_BUILD_OPTIONS=nocheck or run reprotest where
DEB_BUILD_OPTIONS=nocheck is in one of the builds but not the others
(e.g. passing a custom environment variation).


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20250322/796981f5/attachment.sig>