[Qa-jenkins-scm] [Git][qa/jenkins.debian.net][master] 3 commits: reproducible Debian rebuilder prototype: make internal comment more visible

Holger Levsen gitlab at salsa.debian.org
Sun Feb 28 18:24:35 GMT 2021



Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net


Commits:
24bb6a1a by Holger Levsen at 2021-02-28T19:01:23+01:00
reproducible Debian rebuilder prototype: make internal comment more visible

Signed-off-by: Holger Levsen <holger at layer-acht.org>

- - - - -
190416c6 by Holger Levsen at 2021-02-28T19:05:21+01:00
reproducible Debian rebuilder prototype: drop workaround needed for buster's version of sbuild

Signed-off-by: Holger Levsen <holger at layer-acht.org>

- - - - -
9e4ebffc by Holger Levsen at 2021-02-28T19:19:08+01:00
reproducible Debian rebuilder prototype: more cleanups for bullseye's version of debrebuild

Signed-off-by: Holger Levsen <holger at layer-acht.org>

- - - - -


1 changed file:

- bin/reproducible_debian_rebuilder_prototype.sh


Changes:

=====================================
bin/reproducible_debian_rebuilder_prototype.sh
=====================================
@@ -14,6 +14,9 @@ cat << EOF
 ### independently verify that a distributed Debian binary package comes from the source ###
 ### package it's said to be coming from.						###
 ###											###
+### NOTE: we are using rb-debrebuild here which is a code copy from debrebuild from	###
+### 	  devscripts.git - FIXME: use the version from bullseye.			###
+###											###
 ### Once this goal has been achieved we can document these steps. currently the answer	###
 ### to the question "how can I verify installed package X is reproducible?" is: "it's   ###
 ### really complicated" (and sometimes impossible or requires not yet written code).    ###
@@ -149,87 +152,44 @@ fi
 cat $FILE
 mv $FILE ..
 
-# download the source early to fail early if it's not available...
-output_echo "fetching source package $PKG ($VERSION)"
-# just download the source, don't verify it. (keys will expire and be removed from the keyrings)
-# FIXME: debrebuild should download the source code too (--optionally) and verify it matches the one described in .buildinfo file. -> this is #961861
-#        except that this won't work, contrary to what we suggested, the .buildinfo files don't contain hashes of the source package built. bummer!
-# FIXME: debrebuild also lacks a feature to assemble the source for binNMUs, see #961862
-dget --download-only --allow-unauthenticated https://deb.debian.org/debian/pool/main/$POOLPATH/$PKG/${PKG}_$EVERSION.dsc
-dscverify ${PKG}_$EVERSION.dsc || echo "Warning: failed to verify signature, continueing anyway." # FIXME: we can verify most sources in a Debian release, but never all. basically because keys expire. IOW: this is a pretty fundamental basic problem, for which I know no answer, so let's ignore it and move on ;) for now.
-
 # prepare rebuild command
 DEBREBUILD=$(mktemp -t debrebuild-cmd.XXXXXXXX)
-output_echo "trying to debrebuild $PKG ($BINARY_VERSION), which means building instructions how to re-create the build environment as specified in $URLPATH/$FILE"
-# workaround: we are using rb-debrebuild here which is a code copy from debrebuild from devscripts.git
-# using --no-respect-build-path is a workaround until sbuild 0.81 is available. see https://salsa.debian.org/debian/devscripts/-/merge_requests/203#note_192500
-/srv/jenkins/bin/rb-debrebuild --timestamp=metasnap --no-respect-build-path ../$FILE 2>&1 | tee $DEBREBUILD
-
-# FIXME: file a bug like '#955123 debrebuild: please provide --sbuild-output-only option' but with --output-only-base-release
-# (parsing the debrebuild output to gather this information is way to fragile)
-DISTRO=$(tail -1 $DEBREBUILD | grep '^BASE_DIST=' | cut -d '=' -f2)
-case $DISTRO in
-	stretch|buster|bullseye|bookworm|unstable) ;;
-	*)	output_echo "debrebuild failed."
-		if egrep -q 'cannot find .* in dumpavail' $DEBREBUILD ; then
-			#FIXME: file bug, debrebuild should fail clearly on this (and not this subtile)
-			echo "The following build-dependencies are not available on snapshot.debian.org:"
-			echo
-			egrep 'cannot find .* in dumpavail' $DEBREBUILD
-			echo
-			echo "Warning: this should not happen and it's unclear why it did: $(egrep 'cannot find .* in dumpavail' $DEBREBUILD) - still exiting cleanly as this is out-of-scope."
-			exit 0
-		elif egrep -q 'Cannot locate the following packages via snapshots or the current repo/mirror' $DEBREBUILD ; then
-			echo "Some packages could not be located, check the log above."
-			echo
-			echo "Warning: this should not happen and it's unclear why it did - still exiting cleanly as this is out-of-scope."
-			exit 0
-		else
-			echo "Something went wrong with debrebuild, please take a look what exactly..."
-			echo
-			exit 1
-		fi
-		;;
-esac
-
-# create chroot for sbuild
-if [ -d /schroots/debrebuild-$DISTRO-$ARCH ] ; then
-	output_echo "chroot for $DISTRO/$ARCH exists, good."
-else
-	output_echo "preparing chroot for $DISTRO/$ARCH."
-	# FIXME: "|| true" is dummy code for regenerating this chroot every other week or so
-	sudo sbuild-createchroot $DISTRO /schroots/debrebuild-$DISTRO-$ARCH http://deb.debian.org/debian || true
-
-	# I'm a bit surprised this was needed, as debrebuild has code for this...
-	# FIXME: a bug should probably be filed for this as well
-	echo 'Acquire::Check-Valid-Until "false";' | sudo tee /schroots/debrebuild-$DISTRO-$ARCH/etc/apt/apt.conf.d/23-rebuild
-fi
-
-# actually run sbuild
-# - workaround #955123 in devscripts: debrebuild: please provide --sbuild-output-only option
-#   - using tail
-# - workaround yet unfiled bug in devscript by tail -2 | grep -v BASE_DIST... (see above)
-output_echo "trying to re-sbuild $PKG..."
-SBUILD=$(tail -2 $DEBREBUILD | grep '^SBUILD_CMDLINE=' | cut -d '=' -f2- )
-output_echo "using this sbuild command line:"
-echo "$SBUILD"
-echo
-set +e
-eval "$SBUILD"
-RESULT=$?
-if [ "$RESULT" != "0" ] ; then
-	output_echo "sbuild exited with $RESULT:"
-	LOGFILE="./${PKG}_${VERSION}_${ARCH}.build"
-	if [ -L $LOGFILE ] ; then
-		cat $LOGFILE
-	else
-		output_echo "and no sbuild logfile found:"
-		ls -lart
-	fi
-	output_echo "Warning: sbuild failed. Exiting cleanly as this is out-of-scope."
-	exit 0
-fi
-set -e
+output_echo "trying to debrebuild $PKG ($BINARY_VERSION)"
+/srv/jenkins/bin/rb-debrebuild --builder=mmdebstrap --timestamp=metasnap --buildresults=. ../$FILE 2>&1 | tee $DEBREBUILD
+
+# FIXME: drop if deemed unneeded with bullseye...
+#		if egrep -q 'cannot find .* in dumpavail' $DEBREBUILD ; then
+#			#FIXME: file bug, debrebuild should fail clearly on this (and not this subtile)
+#			echo "The following build-dependencies are not available on snapshot.debian.org:"
+#			echo
+#			egrep 'cannot find .* in dumpavail' $DEBREBUILD
+#			echo
+#			echo "Warning: this should not happen and it's unclear why it did: $(egrep 'cannot find .* in dumpavail' $DEBREBUILD) - still exiting cleanly as this is out-of-scope."
+#			exit 0
+#		elif egrep -q 'Cannot locate the following packages via snapshots or the current repo/mirror' $DEBREBUILD ; then
+#			echo "Some packages could not be located, check the log above."
+#			echo
+#			echo "Warning: this should not happen and it's unclear why it did - still exiting cleanly as this is out-of-scope."
+#			exit 0
+#		else
+#			echo "Something went wrong with debrebuild, please take a look what exactly..."
+#			echo
+#			exit 1
+#		fi
+
+# FIXME: drop if deemed unneeded with bullseye... (also drop the actuall schroot on disk and the sudo rights...)
+## create chroot for sbuild
+#if [ -d /schroots/debrebuild-$DISTRO-$ARCH ] ; then
+#	output_echo "chroot for $DISTRO/$ARCH exists, good."
+#else
+#	output_echo "preparing chroot for $DISTRO/$ARCH."
+#	# FIXME: "|| true" is dummy code for regenerating this chroot every other week or so
+#	sudo sbuild-createchroot $DISTRO /schroots/debrebuild-$DISTRO-$ARCH http://deb.debian.org/debian || true
+#
+#	# I'm a bit surprised this was needed, as debrebuild has code for this...
+#	# FIXME: a bug should probably be filed for this as well
+#	echo 'Acquire::Check-Valid-Until "false";' | sudo tee /schroots/debrebuild-$DISTRO-$ARCH/etc/apt/apt.conf.d/23-rebuild
+#fi
 
 # show what we did/created
 output_echo "File artifacts:"



View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/compare/9ca30125b8294455ad3024add5c84caddf339dba...9e4ebffc4b39d2a4282d382d4c62c8b6bd6d3f26

-- 
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/compare/9ca30125b8294455ad3024add5c84caddf339dba...9e4ebffc4b39d2a4282d382d4c62c8b6bd6d3f26
You're receiving this email because of your account on salsa.debian.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20210228/8362f162/attachment-0001.htm>


More information about the Qa-jenkins-scm mailing list