[Git][qa/jenkins.debian.net][master] i386.reproduce.debian.net: further prepare infom07
Holger Levsen (@holger)
gitlab at salsa.debian.org
Sat Dec 7 19:48:52 GMT 2024
Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net
Commits:
4dd9e243 by Holger Levsen at 2024-12-07T20:48:21+01:00
i386.reproduce.debian.net: further prepare infom07
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
11 changed files:
- bin/common-functions.sh
- + hosts/infom07-i386/etc/dehydrated/conf.d/local_local.sh
- + hosts/infom07-i386/etc/dehydrated/domains.txt
- + hosts/infom07-i386/etc/dehydrated/hooks.sh
- + hosts/infom07-i386/etc/nginx/nginx.conf
- + hosts/infom07-i386/opt/etc/rebuilderd-sync.conf
- + hosts/infom07-i386/opt/etc/rebuilderd-worker.conf
- + hosts/infom07-i386/opt/usr/libexec/rebuilderd/rebuilder-debian.sh
- + hosts/infom07-i386/var/www/html/index.html
- hosts/osuosl5-amd64/etc/dehydrated/domains.txt
- hosts/osuosl5-amd64/var/www/html/index.html
Changes:
=====================================
bin/common-functions.sh
=====================================
@@ -232,7 +232,7 @@ jenkins_zombie_check() {
# this has happened on 2023-08-25 again
# this has happened on 2023-09-06 again
#
- ZOMBIES="$(ls -1d /var/lib/jenkins/jobs/* | grep -E 'strip-nondeterminism|reproducible_(builder_(amd64|i386|armhf|arm64)|setup_(pbuilder|schroot)_testing)|chroot-installation_wheezy|aptdpkg|stretch_install_education-thin-client-server|jessie_multiarch_versionskew|dpkg_stretch_find_trigger_cycles|dpkg_buster_find_trigger_cycles|sid_install_education-services|buster_install_education-services|lvc|chroot-installation_stretch_.*_upgrade_to_sid|chroot-installation_buster_.*_upgrade_to_sid|piuparts_.*_(jessie|stretch|buster|bullseye)|lintian-tests|udd_stretch|d-i_pu-build|debsums-tests_(stretch|buster|bullseye)|debian-archive-keyring-tests_stretch|debian-archive-keyring-tests_buster|debian-archive-keyring-tests_bullseye|chroot-installation_jessie|chroot-installation_.*education-lang-|kirkwoot|rebootstrap_.*_gcc1[0123]($|_)|brcm47xx|rebootstrap_(kfreebsd-|nios2_)|diffoscope_from_git_|disorderfs_from_git_master|diffoscope_pypi|diffoscope_freebsd|diffoscope_netbsd|diffoscope_macports|diffoscope_archlinux|openwrt-target-(ath97|tegra)|profitbricks|pool_buildinfos_suites|g-i-installation|reproducible_compare_Debian_sha1sums|bbx15|cb3a|ff2a|ff2b|jtk1a|jtk1b|odxu4a|odxu4b|odu3a|opi2a|opi2c|p64b|p64c|ar71xx|live_setup_schroot|reproducible_debian_live_build$|live_build_debian_stretch_gnome|chroot-installation_stretch|chroot-installation_bullseye*upgrade_to_sid|rebuilder_prototype|osuosl167|osuosl168|osuosl169|osuosl170|osuosl171|osuosl172|osuosl173|osuosl174|osuosl184fakeroot-foreign|fdroid|reproducible_.*_reproducible?$|health_check_amd64_snapshot|reproducible_.*_stretch_.*|buster_diffoscope_amd64_osuosl3|chroot-installation_buster|udd_buster_multiarch_versionskew|disorderfs_from_git|reprotest_from_git|diffoscope_from_git|reproducible_create_meta_pkg_sets$|reproducible_scheduler$|d-i_overview_kfreebsd|codethink9|codethink1|reproducible_.*buster|jtx|reproducible_setup_pbuilder_*ionos(4|14)' || true)"
+ ZOMBIES="$(ls -1d /var/lib/jenkins/jobs/* | grep -E 'strip-nondeterminism|reproducible_(builder_(amd64|i386|armhf|arm64)|setup_(pbuilder|schroot)_testing)|chroot-installation_wheezy|aptdpkg|stretch_install_education-thin-client-server|jessie_multiarch_versionskew|dpkg_stretch_find_trigger_cycles|dpkg_buster_find_trigger_cycles|sid_install_education-services|buster_install_education-services|lvc|chroot-installation_stretch_.*_upgrade_to_sid|chroot-installation_buster_.*_upgrade_to_sid|piuparts_.*_(jessie|stretch|buster|bullseye)|lintian-tests|udd_stretch|d-i_pu-build|debsums-tests_(stretch|buster|bullseye)|debian-archive-keyring-tests_stretch|debian-archive-keyring-tests_buster|debian-archive-keyring-tests_bullseye|chroot-installation_jessie|chroot-installation_.*education-lang-|kirkwoot|rebootstrap_.*_gcc1[0123]($|_)|brcm47xx|rebootstrap_(kfreebsd-|nios2_)|diffoscope_from_git_|disorderfs_from_git_master|diffoscope_pypi|diffoscope_freebsd|diffoscope_netbsd|diffoscope_macports|diffoscope_archlinux|openwrt-target-(ath97|tegra)|profitbricks|pool_buildinfos_suites|g-i-installation|reproducible_compare_Debian_sha1sums|bbx15|cb3a|ff2a|ff2b|jtk1a|jtk1b|odxu4a|odxu4b|odu3a|opi2a|opi2c|p64b|p64c|ar71xx|live_setup_schroot|reproducible_debian_live_build$|live_build_debian_stretch_gnome|chroot-installation_stretch|chroot-installation_bullseye*upgrade_to_sid|rebuilder_prototype|osuosl167|osuosl168|osuosl169|osuosl170|osuosl171|osuosl172|osuosl173|osuosl174|osuosl184fakeroot-foreign|fdroid|reproducible_.*_reproducible?$|health_check_amd64_snapshot|reproducible_.*_stretch_.*|buster_diffoscope_amd64_osuosl3|chroot-installation_buster|udd_buster_multiarch_versionskew|disorderfs_from_git|reprotest_from_git|diffoscope_from_git|reproducible_create_meta_pkg_sets$|reproducible_scheduler$|d-i_overview_kfreebsd|codethink9|codethink1|reproducible_.*buster|jtx|reproducible_setup_pbuilder_*ionos(4|14)|reproducible_setup_.*infom07' || true)"
if [ -n "$ZOMBIES" ] ; then
DIRTY=true
figlet 'zombies!!!'
=====================================
hosts/infom07-i386/etc/dehydrated/conf.d/local_local.sh
=====================================
@@ -0,0 +1,13 @@
+# shellcheck shell=sh disable=SC2034
+
+CA="https://acme-v02.api.letsencrypt.org/directory"
+#CA="https://acme-staging-v02.api.letsencrypt.org/directory"
+
+RENEW_DAYS="30"
+KEYSIZE="4096"
+PRIVATE_KEY_RENEW="no"
+
+CHALLENGETYPE="http-01"
+HOOK="/etc/dehydrated/hooks.sh"
+
+CONTACT_EMAIL="contact at reproducible-builds.org"
=====================================
hosts/infom07-i386/etc/dehydrated/domains.txt
=====================================
@@ -0,0 +1 @@
+i386.reproduce.debian.net
=====================================
hosts/infom07-i386/etc/dehydrated/hooks.sh
=====================================
@@ -0,0 +1,38 @@
+#!/bin/bash
+
+set -eu
+
+OP=$1
+
+_log () {
+ echo " + ($OP) $*"
+}
+
+reload_apache () {
+ _log "Reloading apache..."
+ sudo apache2ctl graceful
+}
+
+reload_nginx () {
+ _log "Reloading nginx..."
+ sudo systemctl reload nginx
+}
+
+email () {
+ # $1: domain name $6: timestamp of cert creation
+ printf "%s\n\n %s\t%s" \
+ "The following SSL certifcate has just been renewed:" \
+ "$1" "$(date -u -d @"$6")" | \
+ mail -s "R-B SSL certifcate renewed" root
+}
+
+case "$OP" in
+ deploy_cert)
+ shift
+ reload_nginx
+ email "$@"
+ ;;
+ *)
+ ;;
+esac
+
=====================================
hosts/infom07-i386/etc/nginx/nginx.conf
=====================================
@@ -0,0 +1,78 @@
+user www-data;
+worker_processes auto;
+error_log /var/log/nginx/error.log;
+pid /run/nginx.pid;
+
+events {
+ worker_connections 1024;
+}
+
+http {
+ include mime.types;
+ sendfile on;
+ server_tokens off;
+
+ server {
+ server_name i386.reproduce.debian.net;
+ index index.html;
+ root /var/www/html;
+
+ listen 443;
+ #listen 443 ssl;
+ #ssl_certificate /var/lib/dehydrated/certs/i386.reproduce.debian.net/fullchain.pem;
+ #ssl_certificate_key /var/lib/dehydrated/certs/i386.reproduce.debian.net/privkey.pem;
+
+ ssl_session_cache shared:le_nginx_SSL:10m;
+ ssl_session_timeout 1440m;
+ ssl_session_tickets off;
+ ssl_protocols TLSv1.2 TLSv1.3;
+ ssl_prefer_server_ciphers off;
+ ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-C
+ HACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";
+
+
+ location /.well-known/acme-challenge/ {
+ alias /var/lib/dehydrated/acme-challenges/;
+ disable_symlinks off;
+ autoindex off;
+ }
+
+ location / {
+ # First attempt to serve request as file, then
+ # as directory, then fall back to displaying a 404.
+ try_files $uri $uri/ =404;
+ }
+
+
+ location /api/ {
+ proxy_pass http://127.0.0.1:8484;
+ }
+
+ location /stats/ {
+ alias /home/jspricke/public_html/stats/ ;
+ disable_symlinks off;
+ autoindex on;
+ }
+
+ }
+
+
+ server {
+ listen 80;
+ server_name i386.reproduce.debian.net;
+
+ location /.well-known/acme-challenge/ {
+ alias /var/lib/dehydrated/acme-challenges/;
+ disable_symlinks off;
+ autoindex off;
+ }
+
+ location / {
+ return 301 https://$host$request_uri;
+ }
+
+ #return 404;
+ }
+
+
+}
=====================================
hosts/infom07-i386/opt/etc/rebuilderd-sync.conf
=====================================
@@ -0,0 +1,53 @@
+## rebuild all of archlinux core
+[profile."archlinux-core"]
+distro = "archlinux"
+suite = "core"
+architectures = ["x86_64"]
+source = "https://ftp.halifax.rwth-aachen.de/archlinux/$repo/os/$arch"
+
+## rebuild community packages of specific maintainers, or allow-list packages by name.
+## If no filter is set, all packages are imported, if both filters are set the package only
+## has to match one of them to be included.
+#[profile."archlinux-community"]
+#distro = "archlinux"
+#suite = "community"
+#architectures = ["x86_64"]
+#source = "https://ftp.halifax.rwth-aachen.de/archlinux/$repo/os/$arch"
+#maintainers = ["somebody"]
+#pkgs = ["some-pkg", "python-*"]
+#excludes = ["tensorflow*"]
+
+[profile."debian-main"]
+distro = "debian"
+suite = "main"
+architectures = ["i386"]
+#releases = ["buster", "sid"]
+releases = ["trixie"]
+source = "http://deb.debian.org/debian"
+
+[profile."debian-anarchism"]
+distro = "debian"
+suite = "main"
+architectures = ["amd64"]
+releases = ["sid"]
+pkgs = ["anarchism"]
+source = "http://deb.debian.org/debian"
+
+# a set of packages to test certain edge-cases
+[profile."debian-misc"]
+distro = "debian"
+suite = "main"
+architectures = ["amd64"]
+releases = ["sid"]
+pkgs = ["anarchism", "binutils-arm-none-eabi", "libglib2.0-bin", "libglib2.0-dev", "sniffglue", "librust-sniffglue-dev", "dfrs", "librust-dfrs-dev"]
+source = "http://deb.debian.org/debian"
+
+[profile."tails"]
+distro = "tails"
+suite = "stable"
+source = "https://mirrors.wikimedia.org/tails/"
+
+[profile."tails-alpha"]
+distro = "tails"
+suite = "alpha"
+source = "https://mirrors.wikimedia.org/tails/"
=====================================
hosts/infom07-i386/opt/etc/rebuilderd-worker.conf
=====================================
@@ -0,0 +1,33 @@
+## The rebuilderd to connect to
+endpoint = "http://127.0.0.1:8484"
+## The server would either allowlist our key or require a signup secret
+#signup_secret = "your_signup_key"
+
+[build]
+#timeout = 86400 # 24 hours
+## Set a maximum build log limit in bytes (default: none).
+## When reaching this limit the log is truncated but the rebuilder backend is *not* terminated.
+max_bytes = 104857600 # 100 MiB ## 10 MiB is upstream default
+## By default build output is forwarded to stdout/stderr.
+## This can be disabled by settings this to true.
+#silent = true
+
+[diffoscope]
+## Generate and attach diffs with diffoscope when rebuilding
+enabled = true
+## Pass additional arguments to diffoscope. Use wisely, some options might not work well.
+#args = ["--max-container-depth", "2", "--fuzzy-threshold", "0"]
+## Set a timeout in seconds after which diffoscope is terminated (default: 3600)
+timeout = 600 # 10 minutes
+## Set a maximum diffoscope output limit in bytes (default: none).
+## When reaching this limit, diffoscope is terminated and the output is truncated.
+max_bytes = 41943040 # 40 MiB
+
+[backend."archlinux"]
+path = "/usr/libexec/rebuilderd/rebuilder-archlinux.sh"
+
+[backend."debian"]
+path = "/opt/usr/libexec/rebuilderd/rebuilder-debian.sh"
+
+[backend."tails"]
+path = "/usr/libexec/rebuilderd/rebuilder-tails.sh"
=====================================
hosts/infom07-i386/opt/usr/libexec/rebuilderd/rebuilder-debian.sh
=====================================
@@ -0,0 +1,11 @@
+#!/bin/sh
+set -xe
+cd "$(dirname "$1")"
+
+mkdir -p etc/apt
+mkdir -p var/lib/apt/lists/
+echo 'deb-src [signed-by=/usr/share/keyrings/debian-archive-keyring.gpg] https://deb.debian.org/debian trixie main' > etc/apt/sources.list
+apt-get -o Dir=. update
+apt-get -o Dir=. source --download-only "$(basename "$1" | cut -d_ -f1)"
+
+/usr/bin/debrebuild --buildresult="${REBUILDERD_OUTDIR}" --builder=sbuild+unshare --cache=/srv/rebuilderd/cache -- "${1}"
=====================================
hosts/infom07-i386/var/www/html/index.html
=====================================
@@ -0,0 +1,201 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <title>
+ i386.reproduce.debian.net - trixie
+ </title>
+ <style>
+ body {
+ font-family: sans-serif;
+ }
+ h1 {
+ font-size: 24px;
+ }
+
+ #search-results {
+ margin: 20px 0;
+ }
+
+ pre {
+ margin: 0;
+ }
+
+ #search-results pre {
+ margin: 12px 0;
+ }
+
+ .status {
+ font-weight: bold;
+ }
+
+ .good {
+ color: green;
+ }
+
+ .bad {
+ color: red;
+ }
+
+ .unknown {
+ color: #957fff;
+ }
+
+ footer {
+ font-size: small;
+ margin: 30px 0;
+ }
+ </style>
+ <script>
+ document.addEventListener('DOMContentLoaded', function() {
+ // package search
+ function spanWith(text) {
+ let s = document.createElement('span');
+ s.textContent = text;
+ return s;
+ }
+
+ function linkTo(href, text) {
+ let a = document.createElement('a');
+ a.href = href;
+ a.textContent = text;
+ return a;
+ }
+
+ function searchPkgs(query) {
+ let div = document.getElementById('search-results');
+ let url = '/api/v0/pkgs/list?' + new URLSearchParams({
+ name: query,
+ distro: 'debian',
+ });
+ fetch(url)
+ .then(response => response.json())
+ .then(data => {
+ // clear children
+ div.innerHTML = '';
+
+ data.map(pkg => {
+ let build_id = pkg.build_id;
+ let r = document.createElement('pre');
+
+ r.appendChild(spanWith('['));
+ let status = pkg['status'];
+ let statusSpan = spanWith(status);
+ statusSpan.className += ' status';
+ if (status == 'GOOD') {
+ statusSpan.className += ' good';
+ } else if (status == 'BAD') {
+ statusSpan.className += ' bad';
+ } else {
+ statusSpan.className += ' unknown';
+ }
+ r.appendChild(statusSpan);
+ r.appendChild(spanWith(`] ${pkg['name']} ${pkg['version']}\t\t`));
+
+ if (build_id) {
+ r.appendChild(spanWith(' ['));
+ r.appendChild(linkTo(`/api/v0/builds/${build_id}/log`, 'log'));
+ r.appendChild(spanWith(']'));
+ }
+
+ if (pkg.has_attestation) {
+ r.appendChild(spanWith(' ['));
+ r.appendChild(linkTo(`/api/v0/builds/${build_id}/attestation`, 'attestation'));
+ r.appendChild(spanWith(']'));
+ }
+
+ if (pkg.has_diffoscope) {
+ r.appendChild(spanWith(' ['));
+ r.appendChild(linkTo(`/api/v0/builds/${build_id}/diffoscope`, 'diffoscope'));
+ r.appendChild(spanWith(']'));
+ }
+
+ div.appendChild(r);
+ });
+ });
+ }
+
+ if (location.hash) {
+ searchPkgs(location.hash.substr(1));
+ }
+
+ document.getElementById('search').addEventListener('submit', function(e) {
+ e.preventDefault();
+ let query = document.getElementById('search-query').value;
+ location.href = `#${query}`;
+ searchPkgs(query);
+ });
+
+ // display stats
+ function updateStats(data) {
+ let div = document.getElementById('stats');
+ // clear children
+ div.innerHTML = '';
+ // add rows
+ data.map(row => {
+ let r = document.createElement('pre');
+ let key = row[0] + ': ';
+ r.textContent = key.padEnd(20) + row[1];
+ div.appendChild(r);
+ });
+ }
+
+ function fetchStats() {
+ fetch('/api/v0/dashboard')
+ .then(response => response.json())
+ .then(data => {
+ let div = document.getElementById('stats');
+ let main = data['suites']['main'];
+
+ let good = main['good'];
+ let bad = main['bad'];
+ let unknown = main['unknown'];
+
+ updateStats([
+ ['good', good],
+ ['bad', bad],
+ ['unknown', unknown],
+ ['build progress', (100 / (good + unknown + bad) * (good + bad)).toFixed(2) + '%'],
+ ['repro', (100 / (good + bad) * good).toFixed(2) + '%'],
+ ]);
+ });
+ }
+
+ setInterval(fetchStats, 60 * 1000);
+ fetchStats();
+ });
+ </script>
+ </head>
+ <body>
+ <h1>
+ i386.reproduce.debian.net
+ </h1>
+ <h3>
+ trixie
+ </h3>
+
+ <div id="stats">Loading stats...</div>
+
+ <div>
+ <h3>Search</h3>
+ <form id="search">
+ <input type="text" id="search-query" placeholder="2ping">
+ <input type="submit" value="Search">
+ </form>
+ </div>
+ <div id="search-results">
+ </div>
+
+ <footer>
+ pew pew, <a href="https://github.com/kpcyrd/rebuilderd">rebuilderd</a> using <a href="https://tracker.debian.org/pkg/devscripts">debrebuild</a> to reproduce what Debian distributes via <code>deb.debian.org</code>. ♥️
+ </br>
+ <a href="/stats/">Statistics about BAD packages.</a> (coming soon: that script in git.)
+ </br>
+ <a href="https://reproduce.debian.net">https://reproduce.debian.net</a>
+ </br>
+ <small>
+ This setup is in it's infancy. Don't consider any URLs as stable here, and more to the point, especially don't take any numbers here seriously, unless...!?!
+ </small>
+ </footer>
+ </body>
+</html>
=====================================
hosts/osuosl5-amd64/etc/dehydrated/domains.txt
=====================================
@@ -1 +1,2 @@
reproduce.debian.net
+amd64.reproduce.debian.net
=====================================
hosts/osuosl5-amd64/var/www/html/index.html
=====================================
@@ -197,7 +197,7 @@
</br>
coming soon: rebuilderd packaged for Debian.
</br>
- coming soon: rebuilderd for arm64/i386 and more hopefully. Do you have hardware?
+ coming soon: rebuilderd for arm64/<a href="https://i386.reproduce.debian.net/"i386</a> and more hopefully. Do you have hardware?
</br>
<small>
This setup is in it's infancy. Don't consider any URLs as stable here, and more to the point, especially don't take any numbers here seriously, unless...!?!
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/4dd9e2436a6b46a880025f40aa771e2a5f4620a6
--
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/4dd9e2436a6b46a880025f40aa771e2a5f4620a6
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20241207/b4608538/attachment-0001.htm>
More information about the Qa-jenkins-scm
mailing list