[Qa-jenkins-scm] [Git][qa/jenkins.debian.net][master] 3 commits: reproducible Debian rebuilder prototype: make internal comment more visible
Holger Levsen
gitlab at salsa.debian.org
Sun Feb 28 18:24:35 GMT 2021
Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net
Commits:
24bb6a1a by Holger Levsen at 2021-02-28T19:01:23+01:00
reproducible Debian rebuilder prototype: make internal comment more visible
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
190416c6 by Holger Levsen at 2021-02-28T19:05:21+01:00
reproducible Debian rebuilder prototype: drop workaround needed for buster's version of sbuild
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
9e4ebffc by Holger Levsen at 2021-02-28T19:19:08+01:00
reproducible Debian rebuilder prototype: more cleanups for bullseye's version of debrebuild
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
1 changed file:
- bin/reproducible_debian_rebuilder_prototype.sh
Changes:
=====================================
bin/reproducible_debian_rebuilder_prototype.sh
=====================================
@@ -14,6 +14,9 @@ cat << EOF
### independently verify that a distributed Debian binary package comes from the source ###
### package it's said to be coming from. ###
### ###
+### NOTE: we are using rb-debrebuild here which is a code copy from debrebuild from ###
+### devscripts.git - FIXME: use the version from bullseye. ###
+### ###
### Once this goal has been achieved we can document these steps. currently the answer ###
### to the question "how can I verify installed package X is reproducible?" is: "it's ###
### really complicated" (and sometimes impossible or requires not yet written code). ###
@@ -149,87 +152,44 @@ fi
cat $FILE
mv $FILE ..
-# download the source early to fail early if it's not available...
-output_echo "fetching source package $PKG ($VERSION)"
-# just download the source, don't verify it. (keys will expire and be removed from the keyrings)
-# FIXME: debrebuild should download the source code too (--optionally) and verify it matches the one described in .buildinfo file. -> this is #961861
-# except that this won't work, contrary to what we suggested, the .buildinfo files don't contain hashes of the source package built. bummer!
-# FIXME: debrebuild also lacks a feature to assemble the source for binNMUs, see #961862
-dget --download-only --allow-unauthenticated https://deb.debian.org/debian/pool/main/$POOLPATH/$PKG/${PKG}_$EVERSION.dsc
-dscverify ${PKG}_$EVERSION.dsc || echo "Warning: failed to verify signature, continueing anyway." # FIXME: we can verify most sources in a Debian release, but never all. basically because keys expire. IOW: this is a pretty fundamental basic problem, for which I know no answer, so let's ignore it and move on ;) for now.
-
# prepare rebuild command
DEBREBUILD=$(mktemp -t debrebuild-cmd.XXXXXXXX)
-output_echo "trying to debrebuild $PKG ($BINARY_VERSION), which means building instructions how to re-create the build environment as specified in $URLPATH/$FILE"
-# workaround: we are using rb-debrebuild here which is a code copy from debrebuild from devscripts.git
-# using --no-respect-build-path is a workaround until sbuild 0.81 is available. see https://salsa.debian.org/debian/devscripts/-/merge_requests/203#note_192500
-/srv/jenkins/bin/rb-debrebuild --timestamp=metasnap --no-respect-build-path ../$FILE 2>&1 | tee $DEBREBUILD
-
-# FIXME: file a bug like '#955123 debrebuild: please provide --sbuild-output-only option' but with --output-only-base-release
-# (parsing the debrebuild output to gather this information is way to fragile)
-DISTRO=$(tail -1 $DEBREBUILD | grep '^BASE_DIST=' | cut -d '=' -f2)
-case $DISTRO in
- stretch|buster|bullseye|bookworm|unstable) ;;
- *) output_echo "debrebuild failed."
- if egrep -q 'cannot find .* in dumpavail' $DEBREBUILD ; then
- #FIXME: file bug, debrebuild should fail clearly on this (and not this subtile)
- echo "The following build-dependencies are not available on snapshot.debian.org:"
- echo
- egrep 'cannot find .* in dumpavail' $DEBREBUILD
- echo
- echo "Warning: this should not happen and it's unclear why it did: $(egrep 'cannot find .* in dumpavail' $DEBREBUILD) - still exiting cleanly as this is out-of-scope."
- exit 0
- elif egrep -q 'Cannot locate the following packages via snapshots or the current repo/mirror' $DEBREBUILD ; then
- echo "Some packages could not be located, check the log above."
- echo
- echo "Warning: this should not happen and it's unclear why it did - still exiting cleanly as this is out-of-scope."
- exit 0
- else
- echo "Something went wrong with debrebuild, please take a look what exactly..."
- echo
- exit 1
- fi
- ;;
-esac
-
-# create chroot for sbuild
-if [ -d /schroots/debrebuild-$DISTRO-$ARCH ] ; then
- output_echo "chroot for $DISTRO/$ARCH exists, good."
-else
- output_echo "preparing chroot for $DISTRO/$ARCH."
- # FIXME: "|| true" is dummy code for regenerating this chroot every other week or so
- sudo sbuild-createchroot $DISTRO /schroots/debrebuild-$DISTRO-$ARCH http://deb.debian.org/debian || true
-
- # I'm a bit surprised this was needed, as debrebuild has code for this...
- # FIXME: a bug should probably be filed for this as well
- echo 'Acquire::Check-Valid-Until "false";' | sudo tee /schroots/debrebuild-$DISTRO-$ARCH/etc/apt/apt.conf.d/23-rebuild
-fi
-
-# actually run sbuild
-# - workaround #955123 in devscripts: debrebuild: please provide --sbuild-output-only option
-# - using tail
-# - workaround yet unfiled bug in devscript by tail -2 | grep -v BASE_DIST... (see above)
-output_echo "trying to re-sbuild $PKG..."
-SBUILD=$(tail -2 $DEBREBUILD | grep '^SBUILD_CMDLINE=' | cut -d '=' -f2- )
-output_echo "using this sbuild command line:"
-echo "$SBUILD"
-echo
-set +e
-eval "$SBUILD"
-RESULT=$?
-if [ "$RESULT" != "0" ] ; then
- output_echo "sbuild exited with $RESULT:"
- LOGFILE="./${PKG}_${VERSION}_${ARCH}.build"
- if [ -L $LOGFILE ] ; then
- cat $LOGFILE
- else
- output_echo "and no sbuild logfile found:"
- ls -lart
- fi
- output_echo "Warning: sbuild failed. Exiting cleanly as this is out-of-scope."
- exit 0
-fi
-set -e
+output_echo "trying to debrebuild $PKG ($BINARY_VERSION)"
+/srv/jenkins/bin/rb-debrebuild --builder=mmdebstrap --timestamp=metasnap --buildresults=. ../$FILE 2>&1 | tee $DEBREBUILD
+
+# FIXME: drop if deemed unneeded with bullseye...
+# if egrep -q 'cannot find .* in dumpavail' $DEBREBUILD ; then
+# #FIXME: file bug, debrebuild should fail clearly on this (and not this subtile)
+# echo "The following build-dependencies are not available on snapshot.debian.org:"
+# echo
+# egrep 'cannot find .* in dumpavail' $DEBREBUILD
+# echo
+# echo "Warning: this should not happen and it's unclear why it did: $(egrep 'cannot find .* in dumpavail' $DEBREBUILD) - still exiting cleanly as this is out-of-scope."
+# exit 0
+# elif egrep -q 'Cannot locate the following packages via snapshots or the current repo/mirror' $DEBREBUILD ; then
+# echo "Some packages could not be located, check the log above."
+# echo
+# echo "Warning: this should not happen and it's unclear why it did - still exiting cleanly as this is out-of-scope."
+# exit 0
+# else
+# echo "Something went wrong with debrebuild, please take a look what exactly..."
+# echo
+# exit 1
+# fi
+
+# FIXME: drop if deemed unneeded with bullseye... (also drop the actuall schroot on disk and the sudo rights...)
+## create chroot for sbuild
+#if [ -d /schroots/debrebuild-$DISTRO-$ARCH ] ; then
+# output_echo "chroot for $DISTRO/$ARCH exists, good."
+#else
+# output_echo "preparing chroot for $DISTRO/$ARCH."
+# # FIXME: "|| true" is dummy code for regenerating this chroot every other week or so
+# sudo sbuild-createchroot $DISTRO /schroots/debrebuild-$DISTRO-$ARCH http://deb.debian.org/debian || true
+#
+# # I'm a bit surprised this was needed, as debrebuild has code for this...
+# # FIXME: a bug should probably be filed for this as well
+# echo 'Acquire::Check-Valid-Until "false";' | sudo tee /schroots/debrebuild-$DISTRO-$ARCH/etc/apt/apt.conf.d/23-rebuild
+#fi
# show what we did/created
output_echo "File artifacts:"
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/compare/9ca30125b8294455ad3024add5c84caddf339dba...9e4ebffc4b39d2a4282d382d4c62c8b6bd6d3f26
--
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/compare/9ca30125b8294455ad3024add5c84caddf339dba...9e4ebffc4b39d2a4282d382d4c62c8b6bd6d3f26
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20210228/8362f162/attachment-0001.htm>
More information about the Qa-jenkins-scm
mailing list