Bug#1067160: python3-debian: Difficult to iterate over next-level index files from Release

[Colin Watson]

Package: python3-debian
Version: 0.1.49
Severity: wishlist

I've trying to figure out how to build a specialized repository
mirroring tool as part of debusine
(https://freexian-team.pages.debian.net/debusine/), and I started by
looking at python-debian.  First I ran into
https://bugs.debian.org/710923, which would be an issue - past
experience with https://bugs.debian.org/918304 suggests that that's much
harder to get right than it seems.  But, assuming that that could be
solved somehow, I then started thinking about what would be needed next.

After verifying the Release file, a mirroring tool needs to fetch the
next level of index files (typically Packages and Sources).
python-debian offers very little help with this - you get a parsed
version of the various checksums fields, but that's about it - and there
are some non-trivial complications here.  For instance,
https://deb.debian.org/debian/dists/unstable/InRelease lists checksums
for main/binary-amd64/Packages, but those are the checksums you get
after fetching a compressed version of the file and uncompressing it;
the uncompressed version isn't actually on the mirror, and you have to
fetch
https://deb.debian.org/debian/dists/unstable/main/binary-amd64/Packages.xz
or similar instead.  While I have a rough idea of how to implement this
and could borrow code from apt, I don't really want to have to write
more versions of this than necessary.

Ideally (for me), Release would have some way to list the next-level
index files, and provide a way to fetch checksum-verified versions of
them given a base URL, parsed into Packages/Sources instances as
appropriate, and abstracting over all the details of compression
algorithms and such.

I think I may end up using apt or python-apt for now, but python-apt is
always a slightly awkward dependency in a Python codebase due to its
tight coupling with apt, and python-debian is much more convenient in
those terms.

Thanks,

-- 
Colin Watson (he/him)                              [cjwatson at debian.org]