idempotent rebuilds

[Simon Josefsson]

Holger Levsen <holger-Yq+U+nADzX/cAl+TvhycwA at public.gmane.org> writes:

> Hi,
>
> I've not much to add except...
>
> On Wed, Dec 04, 2024 at 10:05:38PM +0100, Simon Josefsson wrote:
>> Another feature idempotent rebuilds may help with is to re-bootstrap the
>> entire Debian trixie+X release from another operating system like Guix
>> or macOS.  Rebuilding all packages in trixie+X from Guix directly is
>> simpler than rebuilding all reverse build dependencies of trixie+X and
>> then building trixie+X using those reverse build dependencies.
>
> I believe rebootstraping Debian via Guix, and then "just" bootstrapping
> trixie, might be a worth trying/doing *and* probably for a variety of reasons
> will be much simpler than achieving idempotent rebuilds.

That is what I had in mind -- and I think idempotent rebuilds are
necessary to complete that bootstrap.  Otherwise the Debian trixie
packages built from such a Guix bootstrap environment won't be identical
to what Debian publish, which ought to be the goal of a rebootstrapping
effort.  Am I missing something?

I think bootstrapping Debian could be reduced to:

1) Create a mechanism to bootstrap Debian trixie(+X) from Guix.  Has
anyone worked on this?  If we assemble a debootstrap environment from
Guix binaries and provide a dpkg-buildpackage binary built from sources
in Guix, then one could start building Debian packages from Debian
source code in that Guix debootstrap chroot, until we have enough
packages to be able to build the entire Debian.

2) Work on fixing all idempotent rebuild problems in testing.  Otherwise
the packages produced from 1) won't be identical to the packages that
Debian publish.

One alternative way to bootstrap Debian would be the following:

1) Reproducibly build all packages in trixie using the buildinfo files
to identify which earlier source packages were used, chaining all the
way back to really old Debian's and rebuilding all earlier packages.
Assuming Debian packages are reproducable this way, the resulting trixie
rebuilt packages will be identical to what Debian pubslih.

2) Somehow create a build environment to rebuild all those "initial"
seed packages.

If this plan is followed, idempotent rebuilds are pointless and doesn't
matter.  Rebuilt packages will ultimately be identical anyway, since
they are built the exact same way the official packages were.  I don't
like this plan.  It seems too painful and fragile to bootstrap Debian by
rebuilding all of Debian's history.

Even if a "idempotent rebuild" of Debian seems like a challenge, it
feels like a simpler way to reach Debian rebootstrapping.

Idempotent rebuilds also has the nice consequence of casting light on
deep supply-chain attacks where someone hid a vulnerability in a Debian
buster binary package that would still affects compilation of Debian
trixie packages, without anyone noticing.  It is hard to find those
vulnerabilities reliably without idempotent rebuilds.  I'm not saying
these vulnerabilities exists (creating a future-proof attack vector is
beyond most people) -- but I'm saying they are a possibility, and we
don't have any way to prove the negative today.

> but hey, one never knows until one tries! ;)

That is a motivating curse. :)

/Simon
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 255 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20241217/8af90a6f/attachment.sig>