[Debian-salsa-ci] Release team to utilize Salsa CI for Forky quality assurance?

[Paul Gevers]

Hi Otto,

[I started composing this reply before the reply from Adrian, some 
overlap here and there.]

On 1/4/26 07:00, Otto Kekäläinen wrote:
> Thus I wanted to ask if the Release Team has had any internal
> discussions on how to better utilize Salsa CI for ensuring Forky has
> minimal amount of regressions?


I recall you asked this question before. (I didn't bother to search for 
it). Before diving into specifics, let me ask the question: what exactly 
are you trying to achieve with the questions you raise?

> Are there any new quality assurance tools you would like the Salsa CI
> pipeline to run by default on all Debian packages in an effort to
> prompt maintainers to ensure those tests pass?


I think salsa-ci is already trying to track what we use, and more. Maybe 
it's good to be aware of a recent change in britney2 where we made some 
lintian tags blockers: 
https://salsa.debian.org/release-team/britney1/-/blob/master/britney?ref_type=heads#L197

There is bug #944459.

> Are there any plans to incorporate the Salsa CI status in unstable ->
> testing migrations?


No. We need checks on uploaded source packages and installed binary 
package, not on what lives on salsa. E.g. soonish we'll start checking 
reproducibility in the form of rebuilding what's in the archive. salsa 
will never be able to do that before the upload actually happened and 
the buildd's have built the package. In my opinion a dedicated service 
(like we have in the form of reproduce.debian.net) is better than trying 
to extract things from salsa. Also there's no guarantee that the 
uploaded source is bit-by-bit identical as what salsa-ci uses, except 
when tagged by tag2upload.

> The vcswatch tool is tracking the CI pipeline status on Salsa for all
> packages that have a Vcs-Git URL pointing to Salsa. There could be for
> example a rule that if a package is hosted on Salsa, and if the git
> repo is up-to-date with what was uploaded and CI is passing, the
> package could migrate faster from unstable->testing. Alternatively
> there could be a negative rule that adds extra delay to any package
> that is not in sync in git or has no CI or a failing CI. I guess one
> could also justify a ruleset where packages with no CI are considered
> neutral, but if there is a CI and it is failing, the package would get
> extra delay or not migrate from unstable to testing at all as there is
> something that provably regressed.


Which checks in salsa-ci do you see regularly failing that we don't 
already check with the current policies in britney2? How to determine 
"if the git repo is up-to-date with what was uploaded"? CI in salsa 
(AFAIK) can be tuned per repository, in my opinion we would need to know 
what is enabled and what is disabled in the pipeline to judge anything. 
Could salsa export results per check? (I'm currently missing a data 
source for blhc checks, I would be interested in such a scan, but you'd 
need to guarantee it matches the buildd logs, so better run the service 
on the buildd logs, which isn't a good fit for salsa).

I just read salsa-ci.yml and I spotted a check for RC-bugs. That's an 
example of a check we'd want to ignore as we handle RC bugs differently 
(we check for regressions, as we do for autopkgtest for existing 
packages in testing).

> I do however feel that
> the Release Team should in *some* way reward packages and maintainers
> who diligently keep CI green and quality high to foster a culture that
> testing, reading test results and proactively caring about quality is
> valued.


While I believe I understand your view, I don't think it's the task of 
the Release Team to promote salsa-ci. We're focused on CI on the 
archive. And I think what salsa-ci could do (but I believe debusine is 
already preparing for that, so it's a bit double) is do pre-upload 
checks and only upload once the pre-upload CI is green. Even if such 
across all package support materializes, I'm not sure I'd want the RT to 
be driving that, but rather project consensus/decision. And then I still 
think we'd want to check the actual built binaries like we do now.

I think there will always be value in pre-upload checks and post-build 
checks. Just like build testing isn't the same as installed testing 
(autopkgtest). We don't need to choose (except how we're using resources).

Paul

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 585 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/debian-salsa-ci/attachments/20260104/656a162f/attachment.sig>