Bug#1072205: prevent re-using package versions for NMUs

[Ansgar 🙀]

Hi,

On Thu, 2025-04-03 at 00:25 +0200, Johannes Schauer Marin Rodrigues wrote:
> Quoting Ansgar 🙀 (2025-04-02 21:41:39)
> > regarding your analysis: I think you could just scan the .changes files as
> > they list all *.deb files uploaded.> Though very old changes only have MD5
> > hashes.
> > 
> > They can be found in
> > file://mirror.ftp-master.debian.org/srv/ftp-master.debian.org/queue/done
> 
> are those the changes files uploading *.debs produced by the buildds?
> 
> Does that include *.changes files from package builds which only ever affected
> incoming because there was a new build of the same package before the next
> dinstall?
> 
> Looking at those .changes files, it indeed seems that the binNMUs were made by
> the buildds, so those are not maintainer changes files.

All regular uploads to the archive should come with a .changes files
and these should be all, that is, for both maintainer uploads and
buildd uploads. (Note that historic buildd uploads were also signed by
a DD key.)

It does not cover uploads to the security archive, but packages synced
from the security archive to the main archive should be recorded there.

It also does not include rejected packages (e.g., for policy queues
before *-proposed-updates, new or even before). But for your use case
these are probably not relevant.

There is a mechanism in DAK to import packages without .changes files,
but that shouldn't be used much. The only regular use case is adding
source packages for Built-Using to the security archive.

> > Regarding your observation regarding bash not showing up in any Packages
> > index: that can happen for (at least) two reasons. The snapshot service does
> > not retrieve all Packages files.
> 
> You mean the snapshot service was buggy and forgot to retrieve an arm64
> Packages file? Is that likely? Lets have a look:

There is no technical mechanism to ensure files end up in snapshot.d.o.
If the service does not import them on time, they will not end up in
snapshot.d.o; the archive does currently not retain files until the
snapshot service picked them up.

There have been times in the past where snapshot.d.o was not always
reliably importing files, see for example [1].

  [1]: https://lists.debian.org/msgid-search/169097599149.3513878.11315997455981479167@localhost

I've not checked if this was the case for the particular instance you
mentioned.


> > Or the package could have been superseded by a newer version before it was
> > ever published in a dinstall run.
> 
> But hashes of packages that were never published by a dinstall should be part
> of the changes file on coccia, right? That would be really nice! A year of
> changes files seems to be just around 2.6 GB which is really tiny compared to
> the data I was processing before.

Yes, they should be.

Only the .changes files might be even less data: the last years also
include .buildinfo files.

> Can I just scp those tar.xz files from coccia to my local box and process them
> to my heart's content? Is there a catch?

I think that is fine.

The equivalent data from the security archive is not on a DD-accessible
mirror, but if you find it useful, we could probably make them
available to you (at least provide a one-time dump). As this is only
about published packages, there is no need for confidentiality.

Ansgar