[Debian-salsa-ci] Release team to utilize Salsa CI for Forky quality assurance?

[Otto Kekäläinen]

Hi,

I was hoping Release Team members would comment on the idea of using
current vcswatch information to accelerate or slow down
unstable->testing migrations and thus postponed discussing Salsa CI
details to avoid derailing the discussions from what rules to have for
migrations.

Seems that discussion is over, so I will proceed to answer Adrian's
comment about Salsa CI now.

...
> > It is not about the specific tests in Salsa CI, but merely the fact
> > that some maintainers are trying to be diligent and test their
> > packages properly _before_ upload, and some just upload and skip doing
> > proper QA activities. As I wrote at the end of my question I was
> > wondering if the Release Team wants to promote such a culture.
>
> I am not sure there is a clear distinction between "diligent" and
> "waste of time" here.

Initially there is some work to enable Salsa CI and most importantly
fix all issues in the package so that the CI passes, but once done,
the amount of work needed to run Salsa CI is minimal. You simply push
to Salsa and wait to see if you get an email that the pipeline
regressed or not. Or you can check it via Salsa in a browser or via
one of the command-line tools.

There are numerous examples of when Salsa CI caught issues that
maintainers fixed before upload, and there are numerous examples of
packages breaking Debian unstable in vain due to issues that could
easily have been detected with Salsa CI.

To take just one example,
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1071245 describes a
version of gnupg2 was in an uninstallable state, and stopping the work
of everyone trying to build packages that depend on gnupg2. Testing
the final commit by the maintainer before upload would have certainly
been diligent and not require much extra time. The amount of 'wasted
time' for everyone else getting their work disrupted by a broken
Debian unstable was significant.

The maintainer started using Salsa CI and that same package has not
caused Debian-wide breakage since. Note that the maintainer was also
doing frequent uploads to Debian unstable to get QA results. After
adopting Salsa CI and merging
https://salsa.debian.org/debian/gnupg2/-/merge_requests/16 the
maintainer no longer had the need to upload to unstable just to test,
but got the feedback immediately in Salsa for every git push.

> The proper QA happens after uploading to unstable (or experimental),
> and whether having more than one CI in the workflow makes sense is
> something that should IMHO be left to the maintainer.

Looking at the 25 packages you are the primary maintainer for
(https://udd.debian.org/dmd/?email1=bunk%40debian.org&nouploader1=on&nosponsor1=on&email2=&email3=&packages=&ignpackages=&format=html#todo),
I see none of them is using Salsa CI, and actually not even hosted in
Salsa at all.

Thus if the Release Team took my suggestions on how to use the
vcswatch results it would not have any effect on your packages as they
would never be out of sync in git or have any CI results to use for
the rules.

> > > > Are there any new quality assurance tools you would like the Salsa CI
> > > > pipeline to run by default on all Debian packages in an effort to
> > > > prompt maintainers to ensure those tests pass?
> > >
> > > AFAIK your "on all Debian packages" is factually incorrect, with
> > > resource limits in the Salsa CI preventing the build of many packages.
> >
> > I am aware only of qt6-webengine not being able to build on Salsa
> > (https://salsa.debian.org/salsa-ci-team/pipeline/-/issues/506).
>
> If there is a ranking of the packages with the largest build times on
> amd64 somewhere, I doubt qt6-webengine would make it into the TOP 100.
>
> webkit2gtk, LLVM, GCC, OpenJDK, acl2 - that's just random packages
> coming into my mind with much larger build times.

I checked these and none of them have a single pipeline run yet,
meaning they didn't even try to use Salsa CI:
- https://salsa.debian.org/webkit-team/webkit/-/pipelines
- https://salsa.debian.org/pkg-llvm-team/llvm-defaults/-/pipelines
- https://salsa.debian.org/openjdk-team/openjdk/-/pipelines

This package has pipelines disabled:
- https://salsa.debian.org/toolchain-team/gcc

This isn't hosted on Salsa or git at all:
- https://tracker.debian.org/pkg/acl2

If these maintainers actually want to use Salsa CI, I am happy to help
them get it running. There are large packages like the Linux kernel
and Godot that use Salsa CI, so it is possible for those who want to.

> > I also
> > assume the resource issues are easy to solve if enough people report
> > them and ask them to be solved, as the limitations are in policies and
> > not in software or pipeline itself.
>
> My guesstimate would be that your "easy to solve" might end up being at
> least an order of magnitude more CPU power.
>
> RAM and disk space are also issues, you could not build all packages
> on a worker that has only 50 GB disk space.
>
> Speed also matters here, a maintainer might not want to wait a week for
> Salsa CI results and then another week for complete britney CI results.

Sure, there may be some package that can't run on Salsa CI, but also
your description above seems to have a lot of exaggeration like "wait
a week for Salsa CI results" and considering that none of your
packages use Salsa CI I suspect you are not actually asking for help
on how to get Salsa CI running, but just wanting to emphasize that not
all packages use Salsa CI.

I don't think using the vcswatch results depends on all packages being
in git or Salsa or using Salsa CI.

> >...
> > > If something has provably regressed, this should already block testing
> > > migration due to failing also in the britney CI.
> >
> > Again, it's not about the individual jobs. Any maintainer can anyway
> > customize their salsa-ci.yml file. This is about the culture of
> > testing changes in CI _before_ upload, and about maintaining the
> > ability to see from the CI which commit or merge request caused the
> > regression. Having this capability and culture most likely leads to
> > higher quality and fewer regressions to fix at release time.
> >...
>
> I do not understand this claim.
>
> Anything automated CI can find should already be found by the britney CI
> immediately and block testing migration.

Britney is not checking the same things as vcswatch.

Quoting my own rules proposal here for reference:

On Sat, 3 Jan 2026 at 22:00, Otto Kekäläinen wrote:
..
> The vcswatch tool is tracking the CI pipeline status on Salsa for all
> packages that have a Vcs-Git URL pointing to Salsa. There could be for
> example a rule that if a package is hosted on Salsa, and if the git
> repo is up-to-date with what was uploaded and CI is passing, the
> package could migrate faster from unstable->testing. Alternatively
> there could be a negative rule that adds extra delay to any package
> that is not in sync in git or has no CI or a failing CI. I guess one
> could also justify a ruleset where packages with no CI are considered
> neutral, but if there is a CI and it is failing, the package would get
> extra delay or not migrate from unstable to testing at all as there is
> something that provably regressed.