usrmerge testing in our CI

[Vagrant Cascadian]

On 2023-06-15, Chris Lamb wrote:
> Holger Levsen wrote:
>
>> <helmut> as soon as buildds are merged, varying trixie no longer makes 
>>          sense to me in either case
> […]
>> so should we stop testing usrmerge variations at all now?
>
> Thanks for taking this to the list. For 100% clarity, I understand
> Holger's "at all" suffix to mean, "shall we disable the usrmerge
> variation for bookworm as well as trixie?"
>
> Assuming that's the correct interpretation, from the perspective of
> someone patching packages to make them reproducible, it no longer
> makes sense to prepare patches that address any usrmerge issues. At
> the very least, they will not be prioritised by package maintainers.

I still see usrmerge as a real-world (even if technically unsupported in
Debian) variation that detects real bugs... though I think even for the
bookworm release cycle, maintainers were hesitant to apply usrmerge
related patches.

Most of the usrmerge bugs I encounter often feel a bit like busywork,
but they are also usually fairly trivial to fix.

Off the top of my head, I do not know how many, but definitely some of
the usrmerge related bugs I have found will successfully build in a
usrmerge environment, but in a way that quietly breaks the package
(e.g. junk in manpages or other documentation, entirely missing
documentation, embedding an entirely sensical path, etc.). It would be
hard to systematically find these bugs without testing builds in
usrmerge and non-usrmerge environments. And I am not sure any other
project makes any more sense to do that sort of testing.


> I'm less clear about whether we should cease testing bookworm. It
> doesn't seem right for the CI to claim that various [bookworm]
> packages are "reproducible in bookworm", when the presence of usrmerge
> (or lack thereof) in bookworm means that they can still vary.

I find it a bit weird to change the variations after the release, as
then the stats will gradually change as packages get retested, without
any actual work done to change them... feels a bit too much like moving
the goalposts retroactively.


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20230615/ac3dae78/attachment.sig>