[Git][qa/jenkins.debian.net][master] i386.reproduce.debian.net: further prepare infom07

Holger Levsen (@holger) gitlab at salsa.debian.org
Sat Dec 7 19:48:52 GMT 2024



Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net


Commits:
4dd9e243 by Holger Levsen at 2024-12-07T20:48:21+01:00
i386.reproduce.debian.net: further prepare infom07

Signed-off-by: Holger Levsen <holger at layer-acht.org>

- - - - -


11 changed files:

- bin/common-functions.sh
- + hosts/infom07-i386/etc/dehydrated/conf.d/local_local.sh
- + hosts/infom07-i386/etc/dehydrated/domains.txt
- + hosts/infom07-i386/etc/dehydrated/hooks.sh
- + hosts/infom07-i386/etc/nginx/nginx.conf
- + hosts/infom07-i386/opt/etc/rebuilderd-sync.conf
- + hosts/infom07-i386/opt/etc/rebuilderd-worker.conf
- + hosts/infom07-i386/opt/usr/libexec/rebuilderd/rebuilder-debian.sh
- + hosts/infom07-i386/var/www/html/index.html
- hosts/osuosl5-amd64/etc/dehydrated/domains.txt
- hosts/osuosl5-amd64/var/www/html/index.html


Changes:

=====================================
bin/common-functions.sh
=====================================
@@ -232,7 +232,7 @@ jenkins_zombie_check() {
 	# this has happened on 2023-08-25 again
 	# this has happened on 2023-09-06 again
 	#
-	ZOMBIES="$(ls -1d /var/lib/jenkins/jobs/* | grep -E 'strip-nondeterminism|reproducible_(builder_(amd64|i386|armhf|arm64)|setup_(pbuilder|schroot)_testing)|chroot-installation_wheezy|aptdpkg|stretch_install_education-thin-client-server|jessie_multiarch_versionskew|dpkg_stretch_find_trigger_cycles|dpkg_buster_find_trigger_cycles|sid_install_education-services|buster_install_education-services|lvc|chroot-installation_stretch_.*_upgrade_to_sid|chroot-installation_buster_.*_upgrade_to_sid|piuparts_.*_(jessie|stretch|buster|bullseye)|lintian-tests|udd_stretch|d-i_pu-build|debsums-tests_(stretch|buster|bullseye)|debian-archive-keyring-tests_stretch|debian-archive-keyring-tests_buster|debian-archive-keyring-tests_bullseye|chroot-installation_jessie|chroot-installation_.*education-lang-|kirkwoot|rebootstrap_.*_gcc1[0123]($|_)|brcm47xx|rebootstrap_(kfreebsd-|nios2_)|diffoscope_from_git_|disorderfs_from_git_master|diffoscope_pypi|diffoscope_freebsd|diffoscope_netbsd|diffoscope_macports|diffoscope_archlinux|openwrt-target-(ath97|tegra)|profitbricks|pool_buildinfos_suites|g-i-installation|reproducible_compare_Debian_sha1sums|bbx15|cb3a|ff2a|ff2b|jtk1a|jtk1b|odxu4a|odxu4b|odu3a|opi2a|opi2c|p64b|p64c|ar71xx|live_setup_schroot|reproducible_debian_live_build$|live_build_debian_stretch_gnome|chroot-installation_stretch|chroot-installation_bullseye*upgrade_to_sid|rebuilder_prototype|osuosl167|osuosl168|osuosl169|osuosl170|osuosl171|osuosl172|osuosl173|osuosl174|osuosl184fakeroot-foreign|fdroid|reproducible_.*_reproducible?$|health_check_amd64_snapshot|reproducible_.*_stretch_.*|buster_diffoscope_amd64_osuosl3|chroot-installation_buster|udd_buster_multiarch_versionskew|disorderfs_from_git|reprotest_from_git|diffoscope_from_git|reproducible_create_meta_pkg_sets$|reproducible_scheduler$|d-i_overview_kfreebsd|codethink9|codethink1|reproducible_.*buster|jtx|reproducible_setup_pbuilder_*ionos(4|14)' || true)"
+	ZOMBIES="$(ls -1d /var/lib/jenkins/jobs/* | grep -E 'strip-nondeterminism|reproducible_(builder_(amd64|i386|armhf|arm64)|setup_(pbuilder|schroot)_testing)|chroot-installation_wheezy|aptdpkg|stretch_install_education-thin-client-server|jessie_multiarch_versionskew|dpkg_stretch_find_trigger_cycles|dpkg_buster_find_trigger_cycles|sid_install_education-services|buster_install_education-services|lvc|chroot-installation_stretch_.*_upgrade_to_sid|chroot-installation_buster_.*_upgrade_to_sid|piuparts_.*_(jessie|stretch|buster|bullseye)|lintian-tests|udd_stretch|d-i_pu-build|debsums-tests_(stretch|buster|bullseye)|debian-archive-keyring-tests_stretch|debian-archive-keyring-tests_buster|debian-archive-keyring-tests_bullseye|chroot-installation_jessie|chroot-installation_.*education-lang-|kirkwoot|rebootstrap_.*_gcc1[0123]($|_)|brcm47xx|rebootstrap_(kfreebsd-|nios2_)|diffoscope_from_git_|disorderfs_from_git_master|diffoscope_pypi|diffoscope_freebsd|diffoscope_netbsd|diffoscope_macports|diffoscope_archlinux|openwrt-target-(ath97|tegra)|profitbricks|pool_buildinfos_suites|g-i-installation|reproducible_compare_Debian_sha1sums|bbx15|cb3a|ff2a|ff2b|jtk1a|jtk1b|odxu4a|odxu4b|odu3a|opi2a|opi2c|p64b|p64c|ar71xx|live_setup_schroot|reproducible_debian_live_build$|live_build_debian_stretch_gnome|chroot-installation_stretch|chroot-installation_bullseye*upgrade_to_sid|rebuilder_prototype|osuosl167|osuosl168|osuosl169|osuosl170|osuosl171|osuosl172|osuosl173|osuosl174|osuosl184fakeroot-foreign|fdroid|reproducible_.*_reproducible?$|health_check_amd64_snapshot|reproducible_.*_stretch_.*|buster_diffoscope_amd64_osuosl3|chroot-installation_buster|udd_buster_multiarch_versionskew|disorderfs_from_git|reprotest_from_git|diffoscope_from_git|reproducible_create_meta_pkg_sets$|reproducible_scheduler$|d-i_overview_kfreebsd|codethink9|codethink1|reproducible_.*buster|jtx|reproducible_setup_pbuilder_*ionos(4|14)|reproducible_setup_.*infom07' || true)"
 	if [ -n "$ZOMBIES" ] ; then
 		DIRTY=true
 		figlet 'zombies!!!'


=====================================
hosts/infom07-i386/etc/dehydrated/conf.d/local_local.sh
=====================================
@@ -0,0 +1,13 @@
+# shellcheck shell=sh disable=SC2034
+
+CA="https://acme-v02.api.letsencrypt.org/directory"
+#CA="https://acme-staging-v02.api.letsencrypt.org/directory"
+
+RENEW_DAYS="30"
+KEYSIZE="4096"
+PRIVATE_KEY_RENEW="no"
+
+CHALLENGETYPE="http-01"
+HOOK="/etc/dehydrated/hooks.sh"
+
+CONTACT_EMAIL="contact at reproducible-builds.org"


=====================================
hosts/infom07-i386/etc/dehydrated/domains.txt
=====================================
@@ -0,0 +1 @@
+i386.reproduce.debian.net


=====================================
hosts/infom07-i386/etc/dehydrated/hooks.sh
=====================================
@@ -0,0 +1,38 @@
+#!/bin/bash
+
+set -eu
+
+OP=$1
+
+_log () {
+    echo " + ($OP) $*"
+}
+
+reload_apache () {
+    _log "Reloading apache..."
+    sudo apache2ctl graceful
+}
+
+reload_nginx () {
+    _log "Reloading nginx..."
+    sudo systemctl reload nginx
+}
+
+email () {
+    # $1: domain name $6: timestamp of cert creation
+    printf "%s\n\n    %s\t%s" \
+            "The following SSL certifcate has just been renewed:" \
+            "$1" "$(date -u -d @"$6")" | \
+        mail -s "R-B SSL certifcate renewed" root
+}
+
+case "$OP" in
+    deploy_cert)
+        shift
+        reload_nginx
+        email "$@"
+        ;;
+    *)
+        ;;
+esac
+


=====================================
hosts/infom07-i386/etc/nginx/nginx.conf
=====================================
@@ -0,0 +1,78 @@
+user                   www-data;
+worker_processes       auto;
+error_log              /var/log/nginx/error.log;
+pid                    /run/nginx.pid;
+
+events {
+    worker_connections  1024;
+}
+
+http {
+    include             mime.types;
+    sendfile            on;
+    server_tokens       off;
+
+    server {
+        server_name     i386.reproduce.debian.net;
+        index index.html;
+        root /var/www/html;
+
+        listen 443;
+        #listen 443 ssl;
+        #ssl_certificate /var/lib/dehydrated/certs/i386.reproduce.debian.net/fullchain.pem;
+        #ssl_certificate_key /var/lib/dehydrated/certs/i386.reproduce.debian.net/privkey.pem;
+
+        ssl_session_cache shared:le_nginx_SSL:10m;
+        ssl_session_timeout 1440m;
+        ssl_session_tickets off;
+        ssl_protocols TLSv1.2 TLSv1.3;
+        ssl_prefer_server_ciphers off;
+        ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-C
+        HACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384";
+
+
+        location /.well-known/acme-challenge/ {
+            alias /var/lib/dehydrated/acme-challenges/;
+            disable_symlinks off;
+            autoindex off;
+        }
+
+        location / {
+            # First attempt to serve request as file, then
+            # as directory, then fall back to displaying a 404.
+            try_files $uri $uri/ =404;
+        }
+
+
+        location /api/ {
+            proxy_pass http://127.0.0.1:8484;
+        }
+
+	location /stats/ {
+            alias /home/jspricke/public_html/stats/ ;
+            disable_symlinks off;
+            autoindex on;
+    	}
+
+    }
+
+
+    server {
+        listen          80;
+        server_name     i386.reproduce.debian.net;
+
+        location /.well-known/acme-challenge/ {
+            alias /var/lib/dehydrated/acme-challenges/;
+            disable_symlinks off;
+            autoindex off;
+        }
+
+        location / {
+            return 301 https://$host$request_uri;
+        }
+
+        #return 404;
+    }
+
+
+}


=====================================
hosts/infom07-i386/opt/etc/rebuilderd-sync.conf
=====================================
@@ -0,0 +1,53 @@
+## rebuild all of archlinux core
+[profile."archlinux-core"]
+distro = "archlinux"
+suite = "core"
+architectures = ["x86_64"]
+source = "https://ftp.halifax.rwth-aachen.de/archlinux/$repo/os/$arch"
+
+## rebuild community packages of specific maintainers, or allow-list packages by name.
+## If no filter is set, all packages are imported, if both filters are set the package only
+## has to match one of them to be included.
+#[profile."archlinux-community"]
+#distro = "archlinux"
+#suite = "community"
+#architectures = ["x86_64"]
+#source = "https://ftp.halifax.rwth-aachen.de/archlinux/$repo/os/$arch"
+#maintainers = ["somebody"]
+#pkgs = ["some-pkg", "python-*"]
+#excludes = ["tensorflow*"]
+
+[profile."debian-main"]
+distro = "debian"
+suite = "main"
+architectures = ["i386"]
+#releases = ["buster", "sid"]
+releases = ["trixie"]
+source = "http://deb.debian.org/debian"
+
+[profile."debian-anarchism"]
+distro = "debian"
+suite = "main"
+architectures = ["amd64"]
+releases = ["sid"]
+pkgs = ["anarchism"]
+source = "http://deb.debian.org/debian"
+
+# a set of packages to test certain edge-cases
+[profile."debian-misc"]
+distro = "debian"
+suite = "main"
+architectures = ["amd64"]
+releases = ["sid"]
+pkgs = ["anarchism", "binutils-arm-none-eabi", "libglib2.0-bin", "libglib2.0-dev", "sniffglue", "librust-sniffglue-dev", "dfrs", "librust-dfrs-dev"]
+source = "http://deb.debian.org/debian"
+
+[profile."tails"]
+distro = "tails"
+suite = "stable"
+source = "https://mirrors.wikimedia.org/tails/"
+
+[profile."tails-alpha"]
+distro = "tails"
+suite = "alpha"
+source = "https://mirrors.wikimedia.org/tails/"


=====================================
hosts/infom07-i386/opt/etc/rebuilderd-worker.conf
=====================================
@@ -0,0 +1,33 @@
+## The rebuilderd to connect to
+endpoint = "http://127.0.0.1:8484"
+## The server would either allowlist our key or require a signup secret
+#signup_secret = "your_signup_key"
+
+[build]
+#timeout = 86400 # 24 hours
+## Set a maximum build log limit in bytes (default: none).
+## When reaching this limit the log is truncated but the rebuilder backend is *not* terminated.
+max_bytes = 104857600 # 100 MiB ## 10 MiB is upstream default
+## By default build output is forwarded to stdout/stderr.
+## This can be disabled by settings this to true.
+#silent = true
+
+[diffoscope]
+## Generate and attach diffs with diffoscope when rebuilding
+enabled = true
+## Pass additional arguments to diffoscope. Use wisely, some options might not work well.
+#args = ["--max-container-depth", "2", "--fuzzy-threshold", "0"]
+## Set a timeout in seconds after which diffoscope is terminated (default: 3600)
+timeout = 600 # 10 minutes
+## Set a maximum diffoscope output limit in bytes (default: none).
+## When reaching this limit, diffoscope is terminated and the output is truncated.
+max_bytes = 41943040 # 40 MiB
+
+[backend."archlinux"]
+path = "/usr/libexec/rebuilderd/rebuilder-archlinux.sh"
+
+[backend."debian"]
+path = "/opt/usr/libexec/rebuilderd/rebuilder-debian.sh"
+
+[backend."tails"]
+path = "/usr/libexec/rebuilderd/rebuilder-tails.sh"


=====================================
hosts/infom07-i386/opt/usr/libexec/rebuilderd/rebuilder-debian.sh
=====================================
@@ -0,0 +1,11 @@
+#!/bin/sh
+set -xe
+cd "$(dirname "$1")"
+
+mkdir -p etc/apt
+mkdir -p var/lib/apt/lists/
+echo 'deb-src [signed-by=/usr/share/keyrings/debian-archive-keyring.gpg] https://deb.debian.org/debian trixie main' > etc/apt/sources.list
+apt-get -o Dir=. update
+apt-get -o Dir=. source --download-only "$(basename "$1" | cut -d_ -f1)"
+
+/usr/bin/debrebuild --buildresult="${REBUILDERD_OUTDIR}" --builder=sbuild+unshare --cache=/srv/rebuilderd/cache -- "${1}"


=====================================
hosts/infom07-i386/var/www/html/index.html
=====================================
@@ -0,0 +1,201 @@
+<!DOCTYPE html>
+<html lang="en">
+    <head>
+        <meta charset="utf-8">
+        <title>
+		i386.reproduce.debian.net - trixie
+        </title>
+        <style>
+        body {
+            font-family: sans-serif;
+        }
+        h1 {
+            font-size: 24px;
+        }
+
+        #search-results {
+            margin: 20px 0;
+        }
+
+        pre {
+            margin: 0;
+        }
+
+        #search-results pre {
+            margin: 12px 0;
+        }
+
+        .status {
+            font-weight: bold;
+        }
+
+        .good {
+            color: green;
+        }
+
+        .bad {
+            color: red;
+        }
+
+        .unknown {
+            color: #957fff;
+        }
+
+        footer {
+            font-size: small;
+            margin: 30px 0;
+        }
+        </style>
+        <script>
+        document.addEventListener('DOMContentLoaded', function() {
+            // package search
+            function spanWith(text) {
+                let s = document.createElement('span');
+                s.textContent = text;
+                return s;
+            }
+
+            function linkTo(href, text) {
+                let a = document.createElement('a');
+                a.href = href;
+                a.textContent = text;
+                return a;
+            }
+
+            function searchPkgs(query) {
+                let div = document.getElementById('search-results');
+                let url = '/api/v0/pkgs/list?' + new URLSearchParams({
+                    name: query,
+                    distro: 'debian',
+                });
+                fetch(url)
+                    .then(response => response.json())
+                    .then(data => {
+                        // clear children
+                        div.innerHTML = '';
+
+                        data.map(pkg => {
+                            let build_id = pkg.build_id;
+                            let r = document.createElement('pre');
+
+                            r.appendChild(spanWith('['));
+                            let status = pkg['status'];
+                            let statusSpan = spanWith(status);
+                            statusSpan.className += ' status';
+                            if (status == 'GOOD') {
+                                statusSpan.className += ' good';
+                            } else if (status == 'BAD') {
+                                statusSpan.className += ' bad';
+                            } else {
+                                statusSpan.className += ' unknown';
+                            }
+                            r.appendChild(statusSpan);
+                            r.appendChild(spanWith(`] ${pkg['name']} ${pkg['version']}\t\t`));
+
+                            if (build_id) {
+                                r.appendChild(spanWith(' ['));
+                                r.appendChild(linkTo(`/api/v0/builds/${build_id}/log`, 'log'));
+                                r.appendChild(spanWith(']'));
+                            }
+
+                            if (pkg.has_attestation) {
+                                r.appendChild(spanWith(' ['));
+                                r.appendChild(linkTo(`/api/v0/builds/${build_id}/attestation`, 'attestation'));
+                                r.appendChild(spanWith(']'));
+                            }
+
+                            if (pkg.has_diffoscope) {
+                                r.appendChild(spanWith(' ['));
+                                r.appendChild(linkTo(`/api/v0/builds/${build_id}/diffoscope`, 'diffoscope'));
+                                r.appendChild(spanWith(']'));
+                            }
+
+                            div.appendChild(r);
+                        });
+                    });
+            }
+
+            if (location.hash) {
+                searchPkgs(location.hash.substr(1));
+            }
+
+            document.getElementById('search').addEventListener('submit', function(e) {
+                e.preventDefault();
+                let query = document.getElementById('search-query').value;
+                location.href = `#${query}`;
+                searchPkgs(query);
+            });
+
+            // display stats
+            function updateStats(data) {
+                let div = document.getElementById('stats');
+                // clear children
+                div.innerHTML = '';
+                // add rows
+                data.map(row => {
+                    let r = document.createElement('pre');
+                    let key = row[0] + ': ';
+                    r.textContent = key.padEnd(20) + row[1];
+                    div.appendChild(r);
+                });
+            }
+
+            function fetchStats() {
+                fetch('/api/v0/dashboard')
+                    .then(response => response.json())
+                    .then(data => {
+                        let div = document.getElementById('stats');
+                        let main = data['suites']['main'];
+
+                        let good = main['good'];
+                        let bad = main['bad'];
+                        let unknown = main['unknown'];
+
+                        updateStats([
+                            ['good', good],
+                            ['bad', bad],
+                            ['unknown', unknown],
+                            ['build progress', (100 / (good + unknown + bad) * (good + bad)).toFixed(2) + '%'],
+                            ['repro', (100 / (good + bad) * good).toFixed(2) + '%'],
+                        ]);
+                    });
+            }
+
+            setInterval(fetchStats, 60 * 1000);
+            fetchStats();
+        });
+        </script>
+    </head>
+    <body>
+        <h1>
+		i386.reproduce.debian.net
+        </h1>
+        <h3>
+		trixie
+        </h3>
+
+        <div id="stats">Loading stats...</div>
+
+        <div>
+            <h3>Search</h3>
+            <form id="search">
+                <input type="text" id="search-query" placeholder="2ping">
+                <input type="submit" value="Search">
+            </form>
+        </div>
+        <div id="search-results">
+        </div>
+
+        <footer>
+            pew pew, <a href="https://github.com/kpcyrd/rebuilderd">rebuilderd</a> using <a href="https://tracker.debian.org/pkg/devscripts">debrebuild</a> to reproduce what Debian distributes via <code>deb.debian.org</code>. ♥️
+	    </br>
+	    <a href="/stats/">Statistics about BAD packages.</a> (coming soon: that script in git.)
+	    </br>
+	    <a href="https://reproduce.debian.net">https://reproduce.debian.net</a>
+	    </br>
+	    <small>
+		This setup is in it's infancy. Don't consider any URLs as stable here, and more to the point, especially don't take any numbers here seriously, unless...!?!
+	    </small>
+        </footer>
+    </body>
+</html>


=====================================
hosts/osuosl5-amd64/etc/dehydrated/domains.txt
=====================================
@@ -1 +1,2 @@
 reproduce.debian.net
+amd64.reproduce.debian.net


=====================================
hosts/osuosl5-amd64/var/www/html/index.html
=====================================
@@ -197,7 +197,7 @@
 	    </br>
 	    coming soon: rebuilderd packaged for Debian.
 	    </br>
-	    coming soon: rebuilderd for arm64/i386 and more hopefully. Do you have hardware?
+	    coming soon: rebuilderd for arm64/<a href="https://i386.reproduce.debian.net/"i386</a> and more hopefully. Do you have hardware?
 	    </br>
 	    <small>
 		This setup is in it's infancy. Don't consider any URLs as stable here, and more to the point, especially don't take any numbers here seriously, unless...!?!



View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/4dd9e2436a6b46a880025f40aa771e2a5f4620a6

-- 
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/4dd9e2436a6b46a880025f40aa771e2a5f4620a6
You're receiving this email because of your account on salsa.debian.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20241207/b4608538/attachment-0001.htm>


More information about the Qa-jenkins-scm mailing list