[Git][qa/jenkins.debian.net][master] t.r-b.o/d: WIP: add forky, archive bullseye
Holger Levsen (@holger)
gitlab at salsa.debian.org
Tue Aug 12 15:15:26 BST 2025
Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net
Commits:
5761678f by Holger Levsen at 2025-08-12T16:14:57+02:00
t.r-b.o/d: WIP: add forky, archive bullseye
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
8 changed files:
- TODO.forky
- TODO.trixie
- bin/common-functions.sh
- bin/reproducible.ini
- bin/reproducible_common.sh
- bin/reproducible_db_maintenance.py
- job-cfg/reproducible.yaml
- update_jdn.sh
Changes:
=====================================
TODO.forky
=====================================
@@ -1,42 +1,48 @@
== ToDo for a new Debian release
+this is the stuff which needs to be done when forky has been released.
=== Things to be done for tests.r-b.o:
-* confirm postgres17 cluster is working on ionos7
-** then drop postgres15 cluster on i7: pg_dropcluster 15 main
-* confirm postgres17 cluster is working on jenkins
+* confirm postgres19 cluster is working on ionos7
+** then drop postgres17 cluster on i7: pg_dropcluster 17 main
+* confirm postgres19 cluster is working on jenkins
** backup new cluster
-** then drop postgres15 cluster on jenkins: pg_dropcluster 15 main
+** then drop postgres17 cluster on jenkins: pg_dropcluster 17 main
* tests.r-b.o/debian:
-** pause building: touch `NO-RB-BUILDERS-PLEASE`, shutdown build at service, reproducible_cleanup_nodes.sh and place jenkins in quiet-down.
-** `for d in buildinfo dbd dbdjson dbdtxt logdiffs logs rbuild ; do sudo -u jenkins cp -a "$d"/{trixie,forky} ; done`
-** do a sql copy like the one we did for trixie, see migration number 55 (remember to manually run the db_maintenance script!)
-** create pbuilder setup jobs for forky
-** add forky to the configuration (.py and .sh)
+** pause building:
+*** sudo touch /var/lib/jenkins/NO-RB-BUILDERS-PLEASE
+*** sudo systemctl --user -M jenkins@ stop 'reproducible_build@*.service'
+*** sudo -u jenkins /srv/jenkins/bin/reproducible_cleanup_nodes.sh
+*** go to https://jenkins.debian.net/manage and put jenkins in shutdown mode
+** cd ~jenkins/userContent/reproducible/debian ; `for d in buildinfo dbd dbdjson dbdtxt logdiffs logs rbuild ; do sudo -u jenkins cp -av "$d"/{forky,duke} ; done`
+** do a sql copy like the one we did for trixie, see migration number 55, 57 or 58 (remember to manually run the db_maintenance script!)
+** create pbuilder setup jobs for duke and drop bookworm ones
+** create setup diffpscope schroot job for duke and drop bookworm one
+** add duke to the configuration files bin/reproducible.ini and bin/reproducible_common.sh
** adjust scheduler
-** restart jenkins, run the forky pbuilder and chroot jobs
+** restart jenkins, run the duke pbuilder and chroot jobs
** run the maintenance jobs (to create the chdist)
** once the above jobs are successful, restarts the builders
** adopt whatever suite specific hacks _create_meta_pkg_sets.sh has
-** tweak reproducible_common.sh in three locations to start collect statistics on the daily builds for forky too (and make_graph.py too: (in lines 40 and 54++)
-** create reproducible_(cdeboot|deboot|mmdeb)strap jobs for forky
-** create reproducible_debian_live_build jobs for forky
-** add reproducible_debian_live_build.*forky to reproducible_html_nodes_info.sh
-** adjust reproducible_json.py so that it exports data for forky
-** add forky to the suites in reproducible_pool_buildinfos.sh
-** add forky to ~buildinfo/builtin-pho/dists.txt on ionos7
+** tweak reproducible_common.sh in three locations to start collect statistics on the daily builds for duke too (and make_graph.py too: (in lines 40 and 54++)
+** create reproducible_(cdeboot|deboot|mmdeb)strap jobs for duke and drop bookworm ones
+** create reproducible_debian_live_build jobs for duke and drop bookworm ones
+** add reproducible_debian_live_build.*duke to reproducible_html_nodes_info.sh
+** adjust reproducible_json.py so that it exports data for duke
+** add duke to the suites in reproducible_pool_buildinfos.sh
+** add duke to ~buildinfo/builtin-pho/dists.txt on ionos7
** adjust chroot_installation- jobs
** tweak jenkins-shell-monitor.sh as needed
-** add forky to URL redirects in hosts/jenkins/etc/apache2/reproduciblemap.conf
+** add duke to URL redirects in hosts/jenkins/etc/apache2/reproduciblemap.conf
** adjust multiarch jobs in job-cfg/udd.yaml
-** add chroot-installation forky jobs to build_jenkins_job_health_page in maintenance.sh
-** add forky jobs in jobs/dpkg.yaml to find dpkg trigger cycles, drop trixie ones
+** add chroot-installation duke jobs to build_jenkins_job_health_page in maintenance.sh
+** add duke jobs in jobs/dpkg.yaml to find dpkg trigger cycles, drop trixie ones
** update find_obsolete_transitional_packages.sh and job-cfg/obsolete-transitional.yaml
-** add forky jobs in job-cfg/packages-tests.yaml, drop bullseye ones
-** add forky jobs in job-cfg/piuparts.yaml, drop bullseye ones
+** add duke jobs in job-cfg/packages-tests.yaml, drop bookworm ones
+** add duke jobs in job-cfg/piuparts.yaml, drop bookworm ones
** update suite names in README
=== Things to be done for r.d.n:
=====================================
TODO.trixie
=====================================
@@ -5,12 +5,7 @@
=== Things which have been done already:
* things to do when bookworm been released and trixie is open for development:
-** celebrate
** pause building: touch `NO-RB-BUILDERS-PLEASE`, shutdown build at service, reproducible_cleanup_nodes.sh and place jenkins in quiet-down.
-** `for d in buildinfo dbd dbdjson dbdtxt logdiffs logs rbuild ; do sudo -u jenkins cp -a "$d"/{bookworm,trixie} ; done`
-** do a sql copy like the one we did for bookworm, see migration number 55 (remember to manually run the db_maintenance script!)
-** create pbuilder setup jobs for trixie
-** add trixie to the configuration (.py and .sh)
** adjust scheduler
** restart jenkins, run the trixie pbuilder and chroot jobs
** run the maintenance jobs (to create the chdist)
=====================================
bin/common-functions.sh
=====================================
@@ -237,7 +237,7 @@ jenkins_zombie_check() {
# this has happened on 2025-01 again
# this has happened on 2025-05-13 again
#
- ZOMBIES="$(ls -1d /var/lib/jenkins/jobs/* | grep -E '(strip-nondeterminism|reproducible_(builder_(amd64|i386|armhf|arm64)|setup_(pbuilder|schroot)_testing)|chroot-installation_wheezy|aptdpkg|stretch_install_education-thin-client-server|jessie_multiarch_versionskew|dpkg_stretch_find_trigger_cycles|dpkg_buster_find_trigger_cycles|sid_install_education-services|buster_install_education-services|lvc|chroot-installation_stretch_.*_upgrade_to_sid|chroot-installation_buster_.*_upgrade_to_sid|piuparts_.*_(jessie|stretch|buster|bullseye)|lintian-tests|udd_stretch|d-i_pu-build|debsums-tests_(stretch|buster|bullseye)|debian-archive-keyring-tests_stretch|debian-archive-keyring-tests_buster|debian-archive-keyring-tests_bullseye|chroot-installation_jessie|chroot-installation_.*education-lang-|kirkwoot|rebootstrap_.*_gcc1[0123]($|_)|brcm47xx|rebootstrap_(kfreebsd-|nios2_)|diffoscope_from_git_|disorderfs_from_git_master|diffoscope_pypi|diffoscope_freebsd|diffoscope_netbsd|diffoscope_macports|archlinux|openwrt-target-(ath97|tegra)|profitbricks|pool_buildinfos_suites|g-i-installation|reproducible_compare_Debian_sha1sums|bbx15|cb3a|ff2a|ff2b|jtk1a|jtk1b|odxu4a|odxu4b|odu3a|opi2a|opi2c|p64b|p64c|ar71xx|live_setup_schroot|reproducible_debian_live_build$|live_build_debian_stretch_gnome|chroot-installation_stretch|chroot-installation_bullseye*upgrade_to_sid|rebuilder_prototype|osuosl167|osuosl168|osuosl169|osuosl170|osuosl171|osuosl172|osuosl173|osuosl174|osuosl184|fakeroot-foreign|fdroid|reproducible_.*_reproducible?$|health_check_amd64_snapshot|reproducible_.*_stretch_.*|buster_diffoscope_amd64_osuosl3|chroot-installation_buster|udd_buster_multiarch_versionskew|disorderfs_from_git|reprotest_from_git|diffoscope_from_git|reproducible_create_meta_pkg_sets$|reproducible_scheduler$|d-i_overview_kfreebsd|codethink9|codethink1|reproducible_.*buster|jtx|reproducible_setup_pbuilder_*ionos(4|14)|reproducible_setup_.*infom07|reproducible_setup_pbuilder_*codethink01|reproducible_setup_pbuilder_*codethink02|riscv64-03|riscv64-35|i386_infom07|i386_infom08|scheduler_i386|ionos2|ionos12|ionos6|ionos16|chroot-installation_bullseye|dpkg_bookworm_find_trigger_cycles)' || true)"
+ ZOMBIES="$(ls -1d /var/lib/jenkins/jobs/* | grep -E '(strip-nondeterminism|reproducible_(builder_(amd64|i386|armhf|arm64)|setup_(pbuilder|schroot)_testing)|chroot-installation_wheezy|aptdpkg|stretch_install_education-thin-client-server|jessie_multiarch_versionskew|dpkg_stretch_find_trigger_cycles|dpkg_buster_find_trigger_cycles|sid_install_education-services|buster_install_education-services|lvc|chroot-installation_stretch_.*_upgrade_to_sid|chroot-installation_buster_.*_upgrade_to_sid|piuparts_.*_(jessie|stretch|buster|bullseye)|lintian-tests|udd_stretch|d-i_pu-build|debsums-tests_(stretch|buster|bullseye)|debian-archive-keyring-tests_stretch|debian-archive-keyring-tests_buster|debian-archive-keyring-tests_bullseye|chroot-installation_jessie|chroot-installation_.*education-lang-|kirkwoot|rebootstrap_.*_gcc1[0123]($|_)|brcm47xx|rebootstrap_(kfreebsd-|nios2_)|diffoscope_from_git_|disorderfs_from_git_master|diffoscope_pypi|diffoscope_freebsd|diffoscope_netbsd|diffoscope_macports|archlinux|openwrt-target-(ath97|tegra)|profitbricks|pool_buildinfos_suites|g-i-installation|reproducible_compare_Debian_sha1sums|bbx15|cb3a|ff2a|ff2b|jtk1a|jtk1b|odxu4a|odxu4b|odu3a|opi2a|opi2c|p64b|p64c|ar71xx|live_setup_schroot|reproducible_debian_live_build$|live_build_debian_stretch_gnome|chroot-installation_stretch|chroot-installation_bullseye*upgrade_to_sid|rebuilder_prototype|osuosl167|osuosl168|osuosl169|osuosl170|osuosl171|osuosl172|osuosl173|osuosl174|osuosl184|fakeroot-foreign|fdroid|reproducible_.*_reproducible?$|health_check_amd64_snapshot|reproducible_.*_stretch_.*|buster_diffoscope_amd64_osuosl3|chroot-installation_buster|udd_buster_multiarch_versionskew|disorderfs_from_git|reprotest_from_git|diffoscope_from_git|reproducible_create_meta_pkg_sets$|reproducible_scheduler$|d-i_overview_kfreebsd|codethink9|codethink1|reproducible_.*buster|jtx|reproducible_setup_pbuilder_*ionos(4|14)|reproducible_setup_.*infom07|reproducible_setup_pbuilder_*codethink01|reproducible_setup_pbuilder_*codethink02|riscv64-03|riscv64-35|i386_infom07|i386_infom08|scheduler_i386|ionos2|ionos12|ionos6|ionos16|chroot-installation_bullseye|dpkg_bookworm_find_trigger_cycles|setup_pbuilder_bullseye|setup_schroot_bullseye)' || true)"
if [ -n "$ZOMBIES" ] ; then
DIRTY=true
figlet 'zombies!!!'
=====================================
bin/reproducible.ini
=====================================
@@ -24,9 +24,9 @@ buildinfo = buildinfo
[debian]
distro_root = debian
landing_page = reproducible.html
-suites = unstable trixie bookworm bullseye experimental
-released_suites = bookworm bullseye buster stretch
-archived_suites = buster stretch
+suites = unstable forky trixie bookworm experimental
+released_suites = trixie bookworm bullseye buster stretch
+archived_suites = bullseye buster stretch
archs = amd64 arm64 armhf
defaultsuite = unstable
defaultarch = amd64
=====================================
bin/reproducible_common.sh
=====================================
@@ -37,12 +37,12 @@ CHPATH=/srv/reproducible-results/chdist
mkdir -p "$CHPATH"
# Debian suites being tested
-SUITES="bullseye bookworm trixie unstable experimental"
-TESTING_SUITE="trixie"
+SUITES="bookworm trixie forky unstable experimental"
+TESTING_SUITE="forky"
# released Debian suites (some still tested, some not)
-RELEASED_SUITES="bookworm bullseye buster stretch"
+RELEASED_SUITES="trixie bookworm bullseye buster stretch"
# Debian suites not being tested anymore:
-ARCHIVED_SUITES="buster stretch"
+ARCHIVED_SUITES="bullseye buster stretch"
# Debian architectures being tested
ARCHS="amd64 arm64 armhf"
=====================================
bin/reproducible_db_maintenance.py
=====================================
@@ -824,6 +824,26 @@ schema_updates = {
FROM tr JOIN sr ON tr.name=sr.name
AND tr.architecture=sr.architecture""",
],
+ 58: [ # copy trixie packages (including results) in forky
+ """INSERT INTO sources (name, version, suite, architecture, notify_maintainer, distribution)
+ SELECT name, version, 'forky', architecture, notify_maintainer, distribution
+ FROM sources
+ WHERE suite = 'trixie'""",
+ """WITH tr AS (
+ SELECT id, name, suite, architecture, version
+ FROM sources WHERE suite = 'forky'),
+ sr AS (
+ SELECT s.name, s.architecture, r.version, r.status,
+ r.build_date, r.build_duration, r.node1, r.node2, r.job
+ FROM sources AS s JOIN results AS r ON s.id=r.package_id
+ WHERE s.suite = 'trixie')
+ INSERT INTO results (package_id, version, status, build_date,
+ build_duration, node1, node2, job)
+ SELECT tr.id, sr.version, sr.status, sr.build_date,
+ sr.build_duration, sr.node1, sr.node2, sr.job
+ FROM tr JOIN sr ON tr.name=sr.name
+ AND tr.architecture=sr.architecture""",
+ ],
}
=====================================
job-cfg/reproducible.yaml
=====================================
@@ -412,14 +412,14 @@
- '{name}_{my_task}_{my_arch}_{my_hname}':
my_task:
- - 'setup_pbuilder_bullseye':
- my_description: 'Setup pbuilder for reproducible builds of packages from bullseye'
- my_timed: '23 9 7 * *'
- 'setup_pbuilder_bookworm':
my_description: 'Setup pbuilder for reproducible builds of packages from bookworm'
- my_timed: '23 10 7 * *'
+ my_timed: '23 9 7 * *'
- 'setup_pbuilder_trixie':
my_description: 'Setup pbuilder for reproducible builds of packages from trixie'
+ my_timed: '23 10 7 * *'
+ - 'setup_pbuilder_forky':
+ my_description: 'Setup pbuilder for reproducible builds of packages from forky'
my_timed: '23 11 H/3 * *'
- 'setup_pbuilder_unstable':
my_description: 'Setup pbuilder for reproducible builds of packages from unstable'
@@ -455,21 +455,21 @@
- '{name}_setup_schroot_{my_distro}_{my_arch}_{my_hname}':
my_distro:
- - 'bullseye_diffoscope':
- my_description: 'Setup schroot for running diffoscope in a bullseye environment (this is needed to be able to correctly investigate haskell binaries...)'
- my_timed: '23 0 7 * *'
- my_shell: '/srv/jenkins/bin/schroot-create.sh reproducible reproducible-bullseye-diffoscope bullseye diffoscope locales-all'
- my_disabled: false
- 'bookworm_diffoscope':
my_description: 'Setup schroot for running diffoscope in a bookworm environment (this is needed to be able to correctly investigate haskell binaries...)'
- my_timed: '23 1 7,21 * *'
+ my_timed: '23 0 7 * *'
my_shell: '/srv/jenkins/bin/schroot-create.sh reproducible reproducible-bookworm-diffoscope bookworm diffoscope locales-all'
my_disabled: false
- 'trixie_diffoscope':
my_description: 'Setup schroot for running diffoscope in a trixie environment (this is needed to be able to correctly investigate haskell binaries...)'
- my_timed: '23 1 H/2 * *'
+ my_timed: '23 1 7,21 * *'
my_shell: '/srv/jenkins/bin/schroot-create.sh reproducible reproducible-trixie-diffoscope trixie diffoscope locales-all'
my_disabled: false
+ - 'forky_diffoscope':
+ my_description: 'Setup schroot for running diffoscope in a forky environment (this is needed to be able to correctly investigate haskell binaries...)'
+ my_timed: '23 1 H/2 * *'
+ my_shell: '/srv/jenkins/bin/schroot-create.sh reproducible reproducible-forky-diffoscope forky diffoscope locales-all'
+ my_disabled: false
- 'unstable_diffoscope':
my_description: 'Setup schroot for running diffoscope in a unstable environment (this is needed to be able to correctly investigate haskell binaries...)'
my_timed: '42 0 H/2 * *'
=====================================
update_jdn.sh
=====================================
@@ -441,7 +441,7 @@ if [ -f /etc/debian_version ] ; then
case $HOSTNAME in
ionos7-a*) DEBS="$DEBS
postgresql
- postgresql-15-debversion
+ postgresql-17-debversion
python3-pystache
python3-apt
python3-psycopg2
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/5761678f5b141809537e5fc09198b854ed5ef291
--
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/5761678f5b141809537e5fc09198b854ed5ef291
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20250812/939e5085/attachment-0001.htm>
More information about the Qa-jenkins-scm
mailing list