Including build metadata in packages

[Vagrant Cascadian]

On 2022-02-16, Simon McVittie wrote:
> On Sun, 13 Feb 2022 at 14:13:10 -0800, Vagrant Cascadian wrote:
>> Obviously, this would interfere with any meaningful reproducible builds
>> testing for any package that did something like this. Ideally metadata
>> like this about a build should *not* be included in the .deb files
>> themselves.
>
> Relatedly, I would like to be able to capture some information about
> builds even if (perhaps especially if) the build fails. It might make
> sense to combine that with what you're looking at. It doesn't seem
> ideal that for a successful build, the maintainer can recover detailed
> results via a .deb (at a significant reproducibility cost), but for a
> failing build - perhaps one that fails as a result of test regressions
> - they get no information other than what's in the log. If anything,
> these artifacts seem *more* important for failing builds.

Yes, thanks for bringing up the potential for getting better information
out of failed builds; that is a valueable angle to consider!


>> * Split build metadata into a separate file or archive
>> 
>> Some of the debian-installer packages generate tarballs that are not
>> .deb files and are included in the .changes files when uploading to the
>> archive; making a similar generalized option for other packages to put
>> build metadata into a separate artifact might be workable approach,
>> although this would presumably require toolchain changes in dpkg and dak
>> at the very least, and might take a couple release cycles, which
>> is... well, debian.
>> 
>> The possibility of bundling up .buildinfo files into this metadata too,
>> while taking some changes in relevent dpkg, dak, etc. tooling, might in
>> the long term be worth exploring.
>> 
>> There was a relevent bug report in launchpad:
>> 
>>   https://bugs.launchpad.net/launchpad/+bug/1845159
>> 
>> This seems like the best long-term approach, but pretty much *only* a
>> long-term approach...
>
> I think even if we do one of the other approaches as a stopgap, we'll
> want this in the long term.
>
> There are two approaches that could be taken to this. One is to use
> BYHAND, as Paul Wise already discussed. This would require action from the
> ftp team and dak (I think), but nothing special in sbuild or the buildd
> infrastructure.
>
> However, I'd prefer it if this was output from the build alongside the log,
> instead of being exported via the .changes file, so that failing builds
> can also produce artifacts, to help the maintainer and/or porters to
> figure out why the build failed. This would require action in sbuild and
> the buildd infrastructure, but not in dak, because handling build logs is
> not dak's job (and I don't think handling things like the binutils test
> results should be dak's job either).
>
> Here's a straw-man spec, which I have already prototyped in
> <https://salsa.debian.org/debian/sbuild/-/merge_requests/14>:
>
>     Each whitespace-separated token in the Build-Artifacts field represents
>     a filename pattern in the same simplified shell glob syntax used in
>     "Machine-readable debian/copyright file", version 1.0.
>
>     If the pattern matches a directory, its contents are included in
>     the artifacts, recursively. If a pattern matches another file type,
>     it is included in the artifacts as-is. If a pattern does not match
>     anything, nothing is included in the artifacts: this may be diagnosed
>     with a warning, but is not an error.
>
>     If a pattern matches files outside the build directory, is an absolute
>     path or contains ".." segments,  build tools may exclude those files
>     from the artifacts.
>
>     Build tools should collect the artifacts that match the specified
>     patterns, for example in a compressed tar archive, and make them
>     available alongside the build log for inspection. The artifacts should
>     usually be collected regardless of whether the build succeeds or fails.
>
>     The Build-Artifacts field is not copied into the source package control
>     file (dsc(5)), binary package control file (deb-control(5)),
>     changes file (deb-changes(5)) or any other build results.
>
> (To prototype this without dpkg supporting it, X-Build-Artifacts would be
> appropriate, with none of the XS-, XB-, XC- prefixes.)
>
> For example, a package using Meson with recent debhelper versions would
> typically use:
>
>     Build-Artifacts: obj-*/meson-logs
>
> or a package using recursive Automake might use:
>
>     Build-Artifacts:
>      config.log
>      tests/*.log
>      tests/*.trs
>      tests/reftests/*.png
>
> Does that sound like what you had in mind?

This sounds pretty ideal from a reproducible builds perspective! Thanks
for taking the time to think it out and write it up ideas for a
prototype.


> In practice, if a build currently produces a log file with a name like
> foo_1.2-3_amd64.build, I think it would make sense for it to produce an
> accompanying tarball foo_1.2-3_amd64-artifacts.tar.xz or similar. This
> could be pushed down into dpkg, but I think it might actually make more
> sense in sbuild and pbuilder.

Yeah, if dpkg is producing it, I would imagine the artifacts would have
to end up in the .changes file (so that it would get uploaded along with
the relevent build) and then we're back to reproducible builds
comparisons having to exclude certain files...


> To be useful on buildds, the buildd infrastructure would have to pick up
> these artifacts from sbuild and make them available for download alongside
> the build logs. I don't know the buildd infrastructure, so I'm not
> volunteering for that part.

I don't know it either, but at some point I'll probably need to at least
poke my head into the relevent buildd sources...


live well,
  vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20220507/d3e9589d/attachment.sig>