[Qa-jenkins-scm] [Git][qa/jenkins.debian.net][master] reproducible_debian_live_build: Extend testing

[Holger Levsen (@holger)]

Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net


Commits:
d455e466 by Roland Clobus at 2021-11-21T18:49:19+01:00
reproducible_debian_live_build: Extend testing

Extend the tests to include bookworm and sid.

Reduce the testing frequency for bullseye to monthly.

Document each hook and turn it off when the Debian version is
sufficiently new.

Do not use the freshly built ISO files as artifact. They would be able
to consume too much hard disc space in case of differences.

Signed-off-by: Holger Levsen <holger at layer-acht.org>

- - - - -


2 changed files:

- bin/reproducible_debian_live_build.sh
- job-cfg/reproducible.yaml


Changes:

=====================================
bin/reproducible_debian_live_build.sh
=====================================
@@ -137,6 +137,8 @@ fi
 
 # Add additional hooks, that work around known reproducible issues
 # Note: Keep the hooks in sync with https://wiki.debian.org/ReproducibleInstalls/LiveImages
+
+# Prepare a library for deterministic random behaviour of uuid_generate_random
 cat > config/hooks/normal/1000-reproducible-function-uuid_generate_random.hook.chroot << EOF
 #!/bin/sh
 set -e
@@ -202,6 +204,8 @@ END_OF_SOURCE
 /usr/bin/cc -shared -fPIC unrandomize_uuid_generate_random.c -Wall --pedantic -o /usr/lib/unrandomize_uuid_generate_random.so
 rm -f unrandomize_uuid_generate_random.c
 EOF
+
+# No fix in Debian yet
 cat > config/hooks/normal/1001-reproducible-fontconfig.hook.chroot << EOF
 #!/bin/sh
 set -e
@@ -226,6 +230,10 @@ fi
 
 LD_PRELOAD=/usr/lib/unrandomize_uuid_generate_random.so /usr/bin/fc-cache --force --really-force --system-only --verbose
 EOF
+
+# The mdadm hook is required before bookworm
+case ${DEBIAN_VERSION} in
+  bullseye)
 cat > config/hooks/normal/1002-reproducible-mdadm.hook.chroot << EOF
 #!/bin/sh
 set -e
@@ -247,6 +255,10 @@ then
 fi
 sed -i -e '/# This configuration was auto-generated on/cif [ -z \$SOURCE_DATE_EPOCH ]; then\n  echo "# This configuration was auto-generated on \$(date -R) by mkconf"\nelse\n  echo "# This configuration was auto-generated on \$(date -R --utc -d@\$SOURCE_DATE_EPOCH) by mkconf"\nfi' /usr/share/mdadm/mkconf
 EOF
+  ;;
+esac
+
+# No fix in Debian yet
 cat > config/hooks/normal/1003-reproducible-plymouth.hook.chroot << EOF
 #!/bin/sh
 set -e
@@ -272,6 +284,10 @@ then
 fi
 sed -i -e 's|fc-cache -s|LD_PRELOAD=/usr/lib/unrandomize_uuid_generate_random.so fc-cache|' /usr/share/initramfs-tools/hooks/plymouth
 EOF
+
+# The libxml-sax-perl hook is required before bookworm
+case ${DEBIAN_VERSION} in
+  bullseye)
 cat > config/hooks/normal/1004-reproducible-libxml-sax-perl.hook.chroot << EOF
 #!/bin/sh
 set -e
@@ -298,6 +314,10 @@ update-perl-sax-parsers --remove XML::SAX::Expat
 update-perl-sax-parsers --add XML::SAX::Expat --priority 50
 update-perl-sax-parsers --update
 EOF
+  ;;
+esac
+
+# This could be moved to the default live-build configuration
 cat > config/hooks/normal/9000-cleanup-ucf-backup-files.hook.chroot << EOF
 #!/bin/sh
 set -e
@@ -361,6 +381,11 @@ TIMEOUT="240m"
 DIFFOSCOPE="$(schroot --directory /tmp -c chroot:jenkins-reproducible-${DBDSUITE}-diffoscope diffoscope -- --version 2>&1)"
 TMPDIR=${RESULTSDIR}
 call_diffoscope ${PROJECTNAME} ${CONFIGURATION}/live-image-amd64.hybrid.iso
+
+# Do not publish the ISO images as artifact, they would be able to consume too much disk space
+rm -rf ${RESULTSDIR}/b1
+rm -rf ${RESULTSDIR}/b2
+
 # List the content of the results directory
 PAGE=${CONFIGURATION}.html
 if [ -f "${RESULTSDIR}/${PROJECTNAME}/${CONFIGURATION}/live-image-amd64.hybrid.iso.html" ] ; then


=====================================
job-cfg/reproducible.yaml
=====================================
@@ -235,6 +235,11 @@
           option: category
           categories:
             - debian_live_build
+    logrotate:
+      daysToKeep: '{my_keep_build_days}'
+      numToKeep: 50
+      artifactDaysToKeep: -1
+      artifactNumToKeep: -1
     publishers:
       - logparser: *logparser
       - email: *email
@@ -993,11 +998,18 @@
         - standard
         - xfce
     debian_version:
-        - bullseye
+        - bullseye:
+            my_timed: 'H H 8 * *' # The 8th day of a month, sometime during the day
+            my_keep_build_days: 95 # About 3 months of history
+        - bookworm:
+            my_timed: 'H H * * 0' # Every Sunday, sometime during the day
+            my_keep_build_days: 31 # About one month of history
+        - sid:
+            my_timed: 'H H * * *' # Every day, sometime during the day
+            my_keep_build_days: 10 # About 1.5 week of history
     jobs:
         - 'reproducible_debian_live_build_{imagetype}_{debian_version}':
             my_description: 'Build Debian {imagetype} live image for {debian_version} using live-build'
-            my_timed: '55 11 * * 0'
             my_hname: 'osuosl173'
             my_shell: '/srv/jenkins/bin/jenkins_master_wrapper.sh {imagetype} {debian_version}'
             my_trigger: 'reproducible_html_rsync_remote_results'



View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/d455e466ff2ee389678dd8ec44cefa7630186a6c

-- 
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/d455e466ff2ee389678dd8ec44cefa7630186a6c
You're receiving this email because of your account on salsa.debian.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20211121/2de8cb69/attachment-0001.htm>