[rt.debian.org #8547] Re: Debian RT: snapshot mirror

[Vagrant Cascadian]

On 2021-03-31, Holger Levsen wrote:
> On Sat, Mar 06, 2021 at 11:31:51AM +0100, Frédéric Pierret wrote:
> I'm not sure if you are aware of https://github.com/fepitre/snapshot-mirror
> which is Frédéric's project to create a partial snaphot.d.o mirror so that
> we can continue our work on reproducing Debian bullseye arch:and64 and arch:all
> (these are the partial aspects of said mirror: only useful for reproducing
> bullseye amd64/all).
>
> and while I'm not sure how much Frédéric still needs those raw postgresql
> tables (on which he'll surely respond)... I'd like to add another request:
>
> would it be possible to whitelist his IP 92.188.110.153 so it doesn't get
> throttled when accessing snapshot.d.o?
>
> Currently his work is available at http(s)://debian.notset.fr/snapshot
> and goes back until November 2020. We plan to go back until the oldest 
> package built for bullseye... and then, we plan to add another mirror
> of that data which will have better ressources than Frédéric's mirror @home.

I've heard it now has data as far back as June 2020 now, but the main
limiting factor is the throttling that snapshot.debian.org does. It
looks like we may need to mirror as far back as 2017 to actually get
proper coverage to be able to rebuild all of bullseye... so, at this
rate, it may take a year to get a partial mirror working.

Getting a (partial) mirror of snapshot.debian.org live would reduce the
overall burden on snapshot.debian.org, which is presumably why there is
throttling at all, so seems like an all-around-win.

I'm guessing it's just a time limitation, and an acknowledgement at
least would be much appreciated, but if there are other blockers we
would also love to figure out a way to help!


Looking forward to rebuilding Debian all over again!


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20210414/d9453d8c/attachment.sig>