[Qa-jenkins-scm] [Git][qa/jenkins.debian.net][master] 4 commits: split TODO in TODO and TODO.legacy
Holger Levsen (@holger)
gitlab at salsa.debian.org
Sat Jun 17 23:21:29 BST 2023
Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net
Commits:
cf4f33bf by Holger Levsen at 2023-06-18T00:01:10+02:00
split TODO in TODO and TODO.legacy
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
0813e8ca by Holger Levsen at 2023-06-18T00:04:12+02:00
TODO: we can do backups on osuosl5 (thats a physical host with raid) and drop the (unactioned) todo for netconsoles
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
77c5e3b1 by Holger Levsen at 2023-06-18T00:20:27+02:00
update_jdn: set -o pipefail and update comments
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
99ff468e by Holger Levsen at 2023-06-18T00:21:04+02:00
TODO: shuffle a bit and be more verbose
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
3 changed files:
- TODO
- + TODO.legacy
- update_jdn.sh
Changes:
=====================================
TODO
=====================================
@@ -18,9 +18,18 @@ See link:https://jenkins.debian.net/userContent/about.html["about jenkins.debian
== General ToDo
-* replace amd64 in scripts with $HOSTARCH
-* extend /etc/rc.local to do cleanup of lockfiles
-* run all bash scripts with set -u and set -o pipefail: http://redsymbol.net/articles/unofficial-bash-strict-mode/
+* extend /etc/rc.local to do cleanup of lockfiles:
+** rm /var/cache/pbuilder/*tgz.tmp
+* run all bash scripts with set -eu and set -o pipefail: http://redsymbol.net/articles/unofficial-bash-strict-mode/
+** add -o pipefail to all at once first. that should have less fallout that -u.
+** though -u is also very nice. it will catch typos.
+
+=== nodes at OSUOSL
+
+* mention o4+5 in THANKS and explain usage. mention facebook in past sponsors.
+* mv snapshot.r-b.o from osuosl4 to osuosl5
+** setup xfs on o5, then copy snapshot over
+* rebuilder on o4
=== 2023 things
@@ -32,7 +41,6 @@ See link:https://jenkins.debian.net/userContent/about.html["about jenkins.debian
* maintenance job:
** maybe: rm /var/lib/schroot/unpack/d-i-manual* older than 5 days
** maybe: rm /tmp/mmdebstrap.* older than 3 days
-* split TODO in TODO and TODO.legacy?
* djm:
** option: -r -y => report for year X
** option: -a/--action (default/implicit/optional), requiring $1 $2 $3 params...
@@ -64,372 +72,18 @@ See link:https://jenkins.debian.net/userContent/about.html["about jenkins.debian
** remove "so expect changes for some time." line
** remove/reword "Still very simple statistics"
-=== nodes at OSUOSL
-
-* mention o4+5 in THANKS and explain usage. mention facebook in past sponsors.
-* mv snapshot.r-b.o from osuosl4 to osuosl5
-** setup xfs on o5, then copy snapshot over
-* rebuilder on o4
-* setup netconsoles:
-----
-<guerby> | Ramereth, h01ger netconsole is about the only way to diagnose this kind of issue in my experience
-<guerby> | h01ger, setup rsyslogd on one of the gccserver then it's just one modprobe netconsole
-<guerby> | h01ger, since the machine are on the same LAN
-<guerby> (if not on the same LAN, modprobe netconsole netconsole=+@${IPSRC}/eth0,514@${IPRSYSLOG}/${GWMACADDR} )
-<guerby> where GWMACADDR is 00:11:22... your gateway MAC
-<guerby> saved the day lots of time here :)
-<guerby> found this ext4 bug thanks to it, CVE : https://bugzilla.redhat.com/show_bug.cgi?id=1360968
-<guerby> "Red Hat would like to thank Laurent Guerby for bringing this to our attention." and /me thanks netconsole :)
-<guerby> https://bugzilla.kernel.org/show_bug.cgi?id=107301
-<Ramereth> ya, netconsole is very nice to find odd kernel issues
-<h01ger> | and you recommend to have that running all the time on all the nodes? or only when we suspect probs?
-<Ramereth> we currently have it running all the time on all physical nodes
-----
-
-
==== proper backup
-* FIXME: jenkins.d.n needs to be backed up:
-* '/var/lib/jenkins/jobs' (the results - the configs are in .git)
-* '/var/lib/munin'
-* '/var/log'
-* '/root/' (contains etckeeper.git)
-* '/var/lib/jenkins/reproducible.db' (is backed up manually)
-* '/srv/jenkins.debian.net-scm-sync.git' (is backed up manually)
-* '/var/lib/jenkins/plugins/*.jpi' (can be derived from jdn-scm-sync.git)
-* '/srv/jenkins.debian.net-scm-sync.git'
-* '/etc/.git' and '/etc'
-
-=== jenkins-job-builder related
-
-* yaml could be refactored, lots of duplication in there. this seems to be helpful: http://en.wikipedia.org/wiki/YAML#References (pyyaml which jenkins-job-builder uses supports them)
-
-=== debugging job runs should be made easy
-
-----
- < h01ger> | i think the jenkins-debug-job script should be a python script
- < h01ger> | and j-j-b or another yaml parser can supply job configuration knowledge to that script
- < h01ger> | \o/
- < h01ger> | and that python script can also first determine whether the environment is as needed for the job, and if not, complain verbosely+helpfully and exit
-----
-
-== Improve existing tests
-
-=== tests.reproducible-builds.org
-
-=== Debian reproducible builds
-
-* get rid off "set -x # # to debug diffoscoppe/schroot problems"
-** add check if package to be build has been blacklisted since scheduling and abort
-** on SIGTERM, also cleanup on remote build nodes there! (via ssh &)
-** check rbuild logs for "DIFFOSCOPE='E: Failed to change to directory /tmp: Permission denied' and deal with those
-
-* higher prio:
-** reenable disorderfs setup, check that it *always* unmounts + cleans up nicely
-** pkg pages
-*** new table in pkg/test history page: schedule - if that package is currently scheduled
-*** add link to pkg set(s) if pkg is member of some
-** link pkg sets and issues, that is: at least show packages without issues on pkg set pages, maybe also some issues which need actions (like uninvestigated test failures)
-** notes related:
-*** #786396: classify issue by "toolchain" or "package" fix needed: show bugs which block a bug
-*** new page with annotated packages without categorized issues (and probably without bugs as only note content too, else there are too many)
-*** new page with packages that have notes with comments (which are often useful / contain solutions / low-hanging fruits for newcomers)
-*** new page with notes that doesn't make sense: a.) packages which are reproducible but should not, packages that build but shouldn't, etc.
-*** new page with packages which are reproducible on one arch and unreproducible on another arch (in the same suite, so unstable only atm)
-*** new page with packages which ftbfs on one arch and build fine on another arch (in the same suite, so unstable only atm)
-*** new page with packages which ftbfs in testing but build fine on sid
-*** new page with packages which are orphaned but have a reproducible usertagged patch
-*** new page showing arch all packages which are cross-reproducible, and those which are not
-** new pages: r.d.n/$maintainer-email redirecting to r.d.n/maintainers/unstable/${maintainer-email}.html, showing the unreproducible packages for that address. and a sunny "yay, thank you"-summary for those with only reproducible packages.
-** new page: "open bugs with patches, sorted by maintainers" page and to the navigation, make those NMUable bugs visible
-** improve ftbfs page: list packages without bugs and notes first
-** bin/_html_indexes.py: bugs = get_bugs() # this variable should not be global, else merely importing _html_indexes always queries UDD
-** once firefox 48 is available: revert 1b4dc1b3191e3623a0eeb7cacef80be1ab71d0a2 / grep for _js and remove it…
-
-* lesser prio
-** scheduler: check if there have been more than X failures or depwait in the last Y hours and if so unschedule all packages, disable scheduling and send a mail informing us.
-** pkg sets related:
-*** add new pkg set: torbrowser-build-depends
-*** fix essential set: currently it only has the ones explicitly marked Essential:yes; they and their dependencies make up the full "essential closure set" (sometimes also called pseudo-essential)
-*** replace bin/reproducible_installed_on_debian.org with a proper data provider from DSA, e.g. https://salsa.debian.org/dsa-team/mirror/debian.org/blob/master/debian/control
-** a reproducible_log_grep_by_sql.(py|sh) would be nice, to only grep in packages with a certain status (build in the last X days)
-** database issues
-*** stats_build table should have package ids, not just src+suite+arch as primary key
-*** move "untested" field in stats table too? (as in csv output...)
-** blacklist script should tell if a package was already blacklisted. also proper options should be used...
-** maintenance.sh: delete the history pages once a page has been removed from all suites+archs
-** reproducible.debian.net rename: rgrep all the files…
-** debbindiff2diffoscope rename: do s#dbd#ds#g and s#DBD#DS#g and rename dbd directories?
-** diffoscope needs to be run on the target arch... (or rather: run on a 64bit architecture for 64bit architectures and on 32bit for 32 bit archs), this should probably be doable with a simple i386 chroot on the host (so using qemu-static to run it on armhf should not be needed, probably.)
-** support for arbitrary (to be implemented) Debian-PPAs and external repos, by just giving a source URL
-** once stabilized notification emails should go through the package tracker. The 'build' keyword seems to be the better fit for this. To do so just send the emails to dispatch at tracker.debian.org, setting "X-Distro-Tracker-Package: foo" and "X-Distro-Tracker-Keyword: build". This way people wanting to subscribe to our notification don't need to ask us and can do that by themselves.
-** repo-comparison: check for binaries without source
-** issues: currently only state of amd64 is shown. it would be better to display packages as unreproducible if they are unreproducible on any architecture.
-** include diffoscope run time in log
-* for _service.sh
-** enabling the service in update_jdn
-** maintenance job might want to:
-*** check for running builds using systemctl show & ps fax
-*** check if at least one build is running on a build node
-** use ExecStop to kill remote builds on shutdown
-*** remove cleanup_nodes.sh once this works
-*** add support for disabling archs and for shutdown+respan of workers
-*** add support for starting/stopping workers for specific archs
-* missing variations:
-** 32/64 bit kernel variation on i386
-** prebuilder does (user) group variation like this: https://salsa.debian.org/reproducible-builds/reproducible-misc/blob/master/prebuilder/pbuilderhooks/A02_user
-** variation of $TERM and $COLUMN (and maybe $LINES), unset in the first run, set to "linux" and "77" (and maybe "42") in the 2nd run.
-*** actually TERM is set to "linux" by default already, COLUMN is unset
-** vary order of $PATH entries, see #844500
-** vary the length of the build paths (/build/first vs /build/second), only once the unreproducibilities caused by different build paths are solved
-** vary the init system: sysv and systemd
-** vary SSD/HDD on i386?
-** vary temp dir variables such as TMP/TMPDIR/TEMP/TEMPDIR/MAGICK_TMPDIR
-** maybe vary build with pbuilder and sbuild (but maybe only useful with different setup jobs only…)
-
-==== reproducible Debian armhf
-
-* make systems send mail, use port 465
-* rename all the nodes from $HOSTNAME to $HOSTNAME-armhf-rb ?
-** we could get rid of the links in jenkins.d.n.git/hosts/
-** we could simplify .../hosts/*/etc/munin/munin-node.conf
-
-==== reproducible Debian arm64
-
-* vary DEB_BUILD_OPTS? (NUM_CPU)
-
-==== reproducible Debian installation
-
-* see https://wiki.debian.org/ReproducibleInstalls
-* run this as a new job
-
-==== reproducible coreboot
-
-* add more variations: domain+hostname, uid+gid, USER, UTS namespace
-* build the docs?
-* also build with payloads. x86 use seabios as default, arm boards don't have a default. grub is another payload. and these: bayou coreinfo external filo libpayload nvramcui - and:
-** CONFIG_PAYLOAD_NONE=y
-** CONFIG_PAYLOAD_ELF is not set
-** CONFIG_PAYLOAD_LINUX is not set
-** CONFIG_PAYLOAD_SEABIOS is not set
-** CONFIG_PAYLOAD_FILO is not set
-** CONFIG_PAYLOAD_GRUB2 is not set
-** CONFIG_PAYLOAD_TIANOCORE is not set
-* libreboot ships images, verify those?
-* explain status in plain english
-* use disorderfs for 2nd build
-
-==== reproducible OpenWrt
-
-* add credit for logo/artwork
-* explain status in plain english
-* build path variation
-* incorporate popular third-party ("external feeds") packages?
-* html: build variations are wrong
-* html: git commit output includes garbage
-* html: css: add some space on the left side
-
-==== reproducible NetBSD
-
-* explain status in plain english
-** explain MKREPRO is set to "yes"
-** explain MKREPRO_TIMESTAMP set to $SOURCE_DATE_EPOCH
-* use disorderfs for 2nd build
-
-==== reproducible FreeBSD
-
-* useful improvements:
-** investigate how to use tmpfs on freebsd and build there. see mdmfs(8)
-** find a way to be informed about updates and keep it updated - see 'freebsd-update cron' and 'pkg audit'. The latter is run periodic(8) as part of the nightly root@ emails.
-** modify PATH, uid, gid and USER too and host+domainname as well. The VM is only used for this, so we could change the host+domainname temporaily between builds too.
-** add freebsd vm as node to jenkins and run the script directly there, saves lot of ssh hassle
-** run diffoscope natively
-
-* TODO: random notes, to be moved to README
-** we build the freebsd master branch
-** we build with sudo too
-*** rather not change /usr/obj to be '~jenkins/obj' and build with WITH_INSTALL_AS_USER. also not build in /usr/src. if so, we need to define some variable so we can do so.... but we need a stable path anyway, so whats the point.
-*** maybe build as user in /usr/src...
-* first build world, later build ports (pkg info...)
-
-* document how the freebsd build VM was set up:
-** base 10.1 install following https://www.urbas.eu/freebsd-10-and-profitbricks/
-** modified files:
-*** /etc/rc.conf
-*** /etc/resolv.conf
-*** /boot/loader.conf.local
-** pkg install screen git vim sudo munin-node poudriere
-*** configure /usr/local/etc/munin/munin-node.conf to allow jenkins to access it
-*** configure /usr/local/etc/denyhosts.conf and /etc/hosts.allow and touch /etc/hosts.deniedssh
-** adduser holger
-** adduser jenkins (with bash as default shell)
-** adduser mattia
-** mkdir -p /srv/reproducible-results
-** chown -R jenkins:jenkins /srv/
-
-* system maintenance
-** upgraded the VM:
-*** done with: 'freebsd-update upgrade -r 10.2' as root in screen
-*** and with: 'freebsd-update upgrade -r 10.3'
-*** and with: 'freebsd-update upgrade -r 11.0'
-*** and with: 'freebsd-update upgrade -r 11.1'
-*** and with: 'freebsd-update upgrade -r 11.2' followed by 'pkg-static install pkg ; pkg upgrade'
-*** and with: 'freebsd-update upgrade -r 12.0' followed by 'pkg-static install pkg ; pkg upgrade'
-*** and with: 'freebsd-update upgrade -r 12.1' followed by 'pkg-static install pkg ; pkg upgrade'
-*** and with: 'freebsd-update upgrade -r 12.2' followed by 'pkg-static install pkg ; pkg upgrade'
-*** and with: 'freebsd-update upgrade -r 13.0' followed by 'pkg-static install pkg ; pkg upgrade; pkg install screen git vim sudo munin-node poudriere'
-*** and with: 'freebsd-update upgrade -r 13.1' followed by 'pkg-static install pkg ; pkg upgrade'
-*** and with: 'freebsd-update upgrade -r 13.2' followed by 'pkg-static install pkg ; pkg upgrade'
-
-* online disk resizing howto: https://www.freebsd.org/doc/handbook/disks-growing.html
-
-==== reproducible Fedora
-
-* make sure the pages meet https://fedoraproject.org/wiki/Design/Requirements
- and ask the web design team for help via filing a ticket as described there
-* '/var/cache/mock/fedora-23-x86_64/' has three subdirs we need to handle (put on tmpfs, monitor size, clean sometimes): ccache, root_cache and yum_cache
-* '/var/lib/mock' should be put on /srv/workspace aka tmpfs
-* setup script:
-** mock --clean just uninstalls the chroot but it'll still be rebuilt next time using cache. you can delete the caches from /var/cache/mock/ or touch the mock config
-** is /etc/yum/repos.d/fedora.repo really needed?
-** hosts/osuosl1/etc/yum/repos.d/* is really not sooo good but works…
-* build script
-** cleanup mock cache between two builds: --scrub=all might be too much, but whats sensible (or is it --scrub=all?)?
-** no variations introduced yet:
-*** use '-j$NUM_CPU' and 'NEW_NUM_CPU=$(echo $NUM_CPU-1|bc)'
-*** modify TZ, LANG, LC_ALL, umask
-* other bits:
-** use modified rpmbuild package from dhiru
-** verify gpg signatures (via /etc/mock/)
-** one day we will want to schedule all 17k source packages in fedora…
-* build rawhide too (once fedora-23 builds nicely), releasever=rawhide
-
-* more notes:
-** https://fedoraproject.org/wiki/Using_Mock_to_test_package_builds
-** http://miroslav.suchy.cz/blog/archives/2015/05/28/increase_mock_performance_-_build_packages_in_memory/index.html
-** manually create a fedora chroot using rpm, wget + yum: http://geek.co.il/2010/03/14/how-to-build-a-chroot-jail-environment-for-centos
-
-==== reproducible Arch Linux
-
-* setup_archlinux_schroot job:
-** needs to be made idempotent (currently it removes the schroot at the beginning of the job, instead of creating it elsewhere and replacing it on success at the job end…)
-** use schroot tarballs (gzipped), moves are atomic then
-* only disable cert checking on the node running in the future
-* compare the just built pkg.tar.zst with the one available on the arch mirrors. *then* one can truly say "X% of the Arch Linux packages are reproducible and could bit by bit be reproduced in the real world."
-* maintenance job:
-** check for archlinux schroot sessions which should not be there and delete them. complain if that fails.
-
-* use db - see https://tests.reproducible-builds.org/reproducibledb.htm
-** scheduler.sh:
-*** use asp to update trunk packages?
-** html:
-*** leave all pkg.html files, delete them (much) later, then check all pkg.* files are gone
-*** recreate them newly as index.html
-**** link to https://www.archlinux.org/packages/$repo/x86_64/$pkgname/
-**** provide links to debian pages if same package name exist
-*** create new job to recreate all pkg pages
-*** create json
-** misc:
-*** write into all the stats_ tables
-*** make build.sh respect pacman exit code
-*** comparing versions (in scheduler) is probably needed (though it wuld be better if not): if its not empty it must be higher (due to repo constraints), but it can be even higher than in the repo, because we build trunk
-*** debian uses a table removed_packages, should this be used here too?
-*** debian uses a table manual_scheduler to limit the amount of packages someone can schedule per day...
-*** should breakages job look for archlinux breakages? for which cases? a.) grep "was killed after running into timeout after 30m" pkg.html
-** using notes.git should be next!
-
-* rebuilding against the archlinux repos
-** only needs to be done once using https://github.com/Foxboron/devtools-repro
-** to not (pacman know about trunk and thus) build trunk, we need to configure asp.
-** we'll keep building against repo+trunk as we do now (so that archlinux can also benefit from the QA effects)
-
-* fix build.sh:
-** build2.log doesn't get deleted if build1 fails
-** -> rename build2.log to $version_build2.log (don't include package name...)
-
-* things to be done before enabling more builders:
-** build in /srv/workspace instead of /tmp (once this has been done reduce /tmp size back to 15G)
-
-* build on OSUOSL nodes:
-** update thanks & readme
-
-==== reproducible qubes
-
-* add qubes test on t.r-b.o
-----
- git clone https://github.com/qubesos/qubes-builder
- make get-sources BUILDERCONF=scripts/travis-builder.conf COMPONENTS=installer-qubes-os
- export DIST_DOM0=fc23
- export USE_QUBES_REPO_VERSION=3.2
- export INSTALLER_KICKSTART=/tmp/qubes-installer/conf/travis-iso.ks
-
- make qubes iso BUILDERCONF=scripts/travis-builder.conf VERBOSE=0 COMPONENTS=installer-qubes-os
-----
-* depends: apt install createrepo python-yaml
-* once this iso is being tested, it will be interesting to build the Qubes templates as well, as those images (Qubes templates are images) will be copied on the installation iso. the above iso is a stripped down iso without templates… (and not the real thing)
-
-==== reproducible guix
-
-* there's no "apt-get install", because of non-FHS conformance, but see https://www.gnu.org/software/guix/download/
-* there's a privileged build daemon, which is needed to perform fully isolated builds, see https://www.gnu.org/software/guix/manual/html_node/Build-Environment-Setup.html#Build-Environment-Setup
-* it's a bit of work to set up, but all the steps are documented. the "binary installation" method being the easiest.
-* Manolis wrote:
-----
-There are two ways to install guix, through prebuilt binaries or through
-the source.
-
-*Binary installation:
-
-Go to
-<http://www.gnu.org/software/guix/manual/html_node/Binary-Installation.html>,
-grab the tarball and follow the instructions there.
-
-*Source instalation:
-
-First make sure you have the dependencies mentioned at
-<http://www.gnu.org/software/guix/manual/guix.html#Requirements> installed.
-
-Then download Guix's source from
-ftp://alpha.gnu.org/gnu/guix/guix-0.9.0.tar.gz and use the usual
-./configure && make && make install
-
-After you have Guix built, you need to create the build-users and have
-the guix-daemon run as root, as described here
-<https://www.gnu.org/software/guix/manual/html_node/Build-Environment-Setup.html>.
-
-Keep in mind that the guix-daemon must always run as root.
-
-*Testing if everything works:
-
-Now just run `guix package -i vim` as a non-root user. If it runs
-correctly, Guix is ready for work.
-----
-
-
-=== qa.debian.org*
-
-* udd-versionskew: explain jobs in README
-* udd-versionskew: also provide arch-relative version numbers in output too
-
-=== d-i_manual*
-
-* d-i_check_jobs.sh: check for removed manuals (but with existing jobs) missing
-* svn:trunk/manual/po triggers the full build, should trigger language specific builds.
-* svn:trunk/manual is all thats needed, not whole svn:trunk
-
-=== d-i_build*
-
-* d-i_check_jobs.sh: check for removed package (but with existing jobs) missing
-* build packages using jenkins-debian-glue and not with the custom scripts used today?
-* run scripts/digress/ ?
-
-=== chroot-installation_*
-
-* use schroot for chroot-installation, stop using plain chroot everywhere
-
-== Debian Packaging related
-
-This setup should ideally come as a Debian source package, one far away day maybe.
-
+* probably do the backup on osuosl5:
+* jenkins.d.n needs to be backed up:
+** '/var/lib/jenkins/jobs' (the results - the configs are in .git)
+** '/var/lib/munin'
+** '/var/log'
+** '/root/' (contains etckeeper.git)
+** '/var/lib/jenkins/reproducible.db' (is backed up manually)
+** '/srv/jenkins.debian.net-scm-sync.git' (is backed up manually)
+** '/var/lib/jenkins/plugins/*.jpi' (can be derived from jdn-scm-sync.git)
+** '/srv/jenkins.debian.net-scm-sync.git'
+** '/etc/.git' and '/etc'
// vim: set filetype=asciidoc:
=====================================
TODO.legacy
=====================================
@@ -0,0 +1,335 @@
+== legacy TODO
+
+=== jenkins-job-builder related
+
+* yaml could be refactored, lots of duplication in there. this seems to be helpful: http://en.wikipedia.org/wiki/YAML#References (pyyaml which jenkins-job-builder uses supports them)
+
+=== debugging job runs should be made easy
+
+----
+ < h01ger> | i think the jenkins-debug-job script should be a python script
+ < h01ger> | and j-j-b or another yaml parser can supply job configuration knowledge to that script
+ < h01ger> | \o/
+ < h01ger> | and that python script can also first determine whether the environment is as needed for the job, and if not, complain verbosely+helpfully and exit
+----
+
+== Improve existing tests
+
+=== tests.reproducible-builds.org
+
+=== Debian reproducible builds
+
+* get rid off "set -x # # to debug diffoscoppe/schroot problems"
+** add check if package to be build has been blacklisted since scheduling and abort
+** on SIGTERM, also cleanup on remote build nodes there! (via ssh &)
+** check rbuild logs for "DIFFOSCOPE='E: Failed to change to directory /tmp: Permission denied' and deal with those
+
+* higher prio:
+** reenable disorderfs setup, check that it *always* unmounts + cleans up nicely
+** pkg pages
+*** new table in pkg/test history page: schedule - if that package is currently scheduled
+*** add link to pkg set(s) if pkg is member of some
+** link pkg sets and issues, that is: at least show packages without issues on pkg set pages, maybe also some issues which need actions (like uninvestigated test failures)
+** notes related:
+*** #786396: classify issue by "toolchain" or "package" fix needed: show bugs which block a bug
+*** new page with annotated packages without categorized issues (and probably without bugs as only note content too, else there are too many)
+*** new page with packages that have notes with comments (which are often useful / contain solutions / low-hanging fruits for newcomers)
+*** new page with notes that doesn't make sense: a.) packages which are reproducible but should not, packages that build but shouldn't, etc.
+*** new page with packages which are reproducible on one arch and unreproducible on another arch (in the same suite, so unstable only atm)
+*** new page with packages which ftbfs on one arch and build fine on another arch (in the same suite, so unstable only atm)
+*** new page with packages which ftbfs in testing but build fine on sid
+*** new page with packages which are orphaned but have a reproducible usertagged patch
+*** new page showing arch all packages which are cross-reproducible, and those which are not
+** new pages: r.d.n/$maintainer-email redirecting to r.d.n/maintainers/unstable/${maintainer-email}.html, showing the unreproducible packages for that address. and a sunny "yay, thank you"-summary for those with only reproducible packages.
+** new page: "open bugs with patches, sorted by maintainers" page and to the navigation, make those NMUable bugs visible
+** improve ftbfs page: list packages without bugs and notes first
+** bin/_html_indexes.py: bugs = get_bugs() # this variable should not be global, else merely importing _html_indexes always queries UDD
+** once firefox 48 is available: revert 1b4dc1b3191e3623a0eeb7cacef80be1ab71d0a2 / grep for _js and remove it…
+
+* lesser prio
+** scheduler: check if there have been more than X failures or depwait in the last Y hours and if so unschedule all packages, disable scheduling and send a mail informing us.
+** pkg sets related:
+*** add new pkg set: torbrowser-build-depends
+*** fix essential set: currently it only has the ones explicitly marked Essential:yes; they and their dependencies make up the full "essential closure set" (sometimes also called pseudo-essential)
+*** replace bin/reproducible_installed_on_debian.org with a proper data provider from DSA, e.g. https://salsa.debian.org/dsa-team/mirror/debian.org/blob/master/debian/control
+** a reproducible_log_grep_by_sql.(py|sh) would be nice, to only grep in packages with a certain status (build in the last X days)
+** database issues
+*** stats_build table should have package ids, not just src+suite+arch as primary key
+*** move "untested" field in stats table too? (as in csv output...)
+** blacklist script should tell if a package was already blacklisted. also proper options should be used...
+** maintenance.sh: delete the history pages once a page has been removed from all suites+archs
+** reproducible.debian.net rename: rgrep all the files…
+** debbindiff2diffoscope rename: do s#dbd#ds#g and s#DBD#DS#g and rename dbd directories?
+** diffoscope needs to be run on the target arch... (or rather: run on a 64bit architecture for 64bit architectures and on 32bit for 32 bit archs), this should probably be doable with a simple i386 chroot on the host (so using qemu-static to run it on armhf should not be needed, probably.)
+** support for arbitrary (to be implemented) Debian-PPAs and external repos, by just giving a source URL
+** once stabilized notification emails should go through the package tracker. The 'build' keyword seems to be the better fit for this. To do so just send the emails to dispatch at tracker.debian.org, setting "X-Distro-Tracker-Package: foo" and "X-Distro-Tracker-Keyword: build". This way people wanting to subscribe to our notification don't need to ask us and can do that by themselves.
+** repo-comparison: check for binaries without source
+** issues: currently only state of amd64 is shown. it would be better to display packages as unreproducible if they are unreproducible on any architecture.
+** include diffoscope run time in log
+* for _service.sh
+** enabling the service in update_jdn
+** maintenance job might want to:
+*** check for running builds using systemctl show & ps fax
+*** check if at least one build is running on a build node
+** use ExecStop to kill remote builds on shutdown
+*** remove cleanup_nodes.sh once this works
+*** add support for disabling archs and for shutdown+respan of workers
+*** add support for starting/stopping workers for specific archs
+* missing variations:
+** 32/64 bit kernel variation on i386
+** prebuilder does (user) group variation like this: https://salsa.debian.org/reproducible-builds/reproducible-misc/blob/master/prebuilder/pbuilderhooks/A02_user
+** variation of $TERM and $COLUMN (and maybe $LINES), unset in the first run, set to "linux" and "77" (and maybe "42") in the 2nd run.
+*** actually TERM is set to "linux" by default already, COLUMN is unset
+** vary order of $PATH entries, see #844500
+** vary the length of the build paths (/build/first vs /build/second), only once the unreproducibilities caused by different build paths are solved
+** vary the init system: sysv and systemd
+** vary SSD/HDD on i386?
+** vary temp dir variables such as TMP/TMPDIR/TEMP/TEMPDIR/MAGICK_TMPDIR
+** maybe vary build with pbuilder and sbuild (but maybe only useful with different setup jobs only…)
+
+==== reproducible Debian armhf
+
+* make systems send mail, use port 465
+* rename all the nodes from $HOSTNAME to $HOSTNAME-armhf-rb ?
+** we could get rid of the links in jenkins.d.n.git/hosts/
+** we could simplify .../hosts/*/etc/munin/munin-node.conf
+
+==== reproducible Debian arm64
+
+* vary DEB_BUILD_OPTS? (NUM_CPU)
+
+==== reproducible Debian installation
+
+* see https://wiki.debian.org/ReproducibleInstalls
+* run this as a new job
+
+==== reproducible coreboot
+
+* add more variations: domain+hostname, uid+gid, USER, UTS namespace
+* build the docs?
+* also build with payloads. x86 use seabios as default, arm boards don't have a default. grub is another payload. and these: bayou coreinfo external filo libpayload nvramcui - and:
+** CONFIG_PAYLOAD_NONE=y
+** CONFIG_PAYLOAD_ELF is not set
+** CONFIG_PAYLOAD_LINUX is not set
+** CONFIG_PAYLOAD_SEABIOS is not set
+** CONFIG_PAYLOAD_FILO is not set
+** CONFIG_PAYLOAD_GRUB2 is not set
+** CONFIG_PAYLOAD_TIANOCORE is not set
+* libreboot ships images, verify those?
+* explain status in plain english
+* use disorderfs for 2nd build
+
+==== reproducible OpenWrt
+
+* add credit for logo/artwork
+* explain status in plain english
+* build path variation
+* incorporate popular third-party ("external feeds") packages?
+* html: build variations are wrong
+* html: git commit output includes garbage
+* html: css: add some space on the left side
+
+==== reproducible NetBSD
+
+* explain status in plain english
+** explain MKREPRO is set to "yes"
+** explain MKREPRO_TIMESTAMP set to $SOURCE_DATE_EPOCH
+* use disorderfs for 2nd build
+
+==== reproducible FreeBSD
+
+* useful improvements:
+** investigate how to use tmpfs on freebsd and build there. see mdmfs(8)
+** find a way to be informed about updates and keep it updated - see 'freebsd-update cron' and 'pkg audit'. The latter is run periodic(8) as part of the nightly root@ emails.
+** modify PATH, uid, gid and USER too and host+domainname as well. The VM is only used for this, so we could change the host+domainname temporaily between builds too.
+** add freebsd vm as node to jenkins and run the script directly there, saves lot of ssh hassle
+** run diffoscope natively
+
+* TODO: random notes, to be moved to README
+** we build the freebsd master branch
+** we build with sudo too
+*** rather not change /usr/obj to be '~jenkins/obj' and build with WITH_INSTALL_AS_USER. also not build in /usr/src. if so, we need to define some variable so we can do so.... but we need a stable path anyway, so whats the point.
+*** maybe build as user in /usr/src...
+* first build world, later build ports (pkg info...)
+
+* document how the freebsd build VM was set up:
+** base 10.1 install following https://www.urbas.eu/freebsd-10-and-profitbricks/
+** modified files:
+*** /etc/rc.conf
+*** /etc/resolv.conf
+*** /boot/loader.conf.local
+** pkg install screen git vim sudo munin-node poudriere
+*** configure /usr/local/etc/munin/munin-node.conf to allow jenkins to access it
+*** configure /usr/local/etc/denyhosts.conf and /etc/hosts.allow and touch /etc/hosts.deniedssh
+** adduser holger
+** adduser jenkins (with bash as default shell)
+** adduser mattia
+** mkdir -p /srv/reproducible-results
+** chown -R jenkins:jenkins /srv/
+
+* system maintenance
+** upgraded the VM:
+*** done with: 'freebsd-update upgrade -r 10.2' as root in screen
+*** and with: 'freebsd-update upgrade -r 10.3'
+*** and with: 'freebsd-update upgrade -r 11.0'
+*** and with: 'freebsd-update upgrade -r 11.1'
+*** and with: 'freebsd-update upgrade -r 11.2' followed by 'pkg-static install pkg ; pkg upgrade'
+*** and with: 'freebsd-update upgrade -r 12.0' followed by 'pkg-static install pkg ; pkg upgrade'
+*** and with: 'freebsd-update upgrade -r 12.1' followed by 'pkg-static install pkg ; pkg upgrade'
+*** and with: 'freebsd-update upgrade -r 12.2' followed by 'pkg-static install pkg ; pkg upgrade'
+*** and with: 'freebsd-update upgrade -r 13.0' followed by 'pkg-static install pkg ; pkg upgrade; pkg install screen git vim sudo munin-node poudriere'
+*** and with: 'freebsd-update upgrade -r 13.1' followed by 'pkg-static install pkg ; pkg upgrade'
+*** and with: 'freebsd-update upgrade -r 13.2' followed by 'pkg-static install pkg ; pkg upgrade'
+
+* online disk resizing howto: https://www.freebsd.org/doc/handbook/disks-growing.html
+
+==== reproducible Fedora
+
+* make sure the pages meet https://fedoraproject.org/wiki/Design/Requirements
+ and ask the web design team for help via filing a ticket as described there
+* '/var/cache/mock/fedora-23-x86_64/' has three subdirs we need to handle (put on tmpfs, monitor size, clean sometimes): ccache, root_cache and yum_cache
+* '/var/lib/mock' should be put on /srv/workspace aka tmpfs
+* setup script:
+** mock --clean just uninstalls the chroot but it'll still be rebuilt next time using cache. you can delete the caches from /var/cache/mock/ or touch the mock config
+** is /etc/yum/repos.d/fedora.repo really needed?
+** hosts/osuosl1/etc/yum/repos.d/* is really not sooo good but works…
+* build script
+** cleanup mock cache between two builds: --scrub=all might be too much, but whats sensible (or is it --scrub=all?)?
+** no variations introduced yet:
+*** use '-j$NUM_CPU' and 'NEW_NUM_CPU=$(echo $NUM_CPU-1|bc)'
+*** modify TZ, LANG, LC_ALL, umask
+* other bits:
+** use modified rpmbuild package from dhiru
+** verify gpg signatures (via /etc/mock/)
+** one day we will want to schedule all 17k source packages in fedora…
+* build rawhide too (once fedora-23 builds nicely), releasever=rawhide
+
+* more notes:
+** https://fedoraproject.org/wiki/Using_Mock_to_test_package_builds
+** http://miroslav.suchy.cz/blog/archives/2015/05/28/increase_mock_performance_-_build_packages_in_memory/index.html
+** manually create a fedora chroot using rpm, wget + yum: http://geek.co.il/2010/03/14/how-to-build-a-chroot-jail-environment-for-centos
+
+==== reproducible Arch Linux
+
+* setup_archlinux_schroot job:
+** needs to be made idempotent (currently it removes the schroot at the beginning of the job, instead of creating it elsewhere and replacing it on success at the job end…)
+** use schroot tarballs (gzipped), moves are atomic then
+* only disable cert checking on the node running in the future
+* compare the just built pkg.tar.zst with the one available on the arch mirrors. *then* one can truly say "X% of the Arch Linux packages are reproducible and could bit by bit be reproduced in the real world."
+* maintenance job:
+** check for archlinux schroot sessions which should not be there and delete them. complain if that fails.
+
+* use db - see https://tests.reproducible-builds.org/reproducibledb.htm
+** scheduler.sh:
+*** use asp to update trunk packages?
+** html:
+*** leave all pkg.html files, delete them (much) later, then check all pkg.* files are gone
+*** recreate them newly as index.html
+**** link to https://www.archlinux.org/packages/$repo/x86_64/$pkgname/
+**** provide links to debian pages if same package name exist
+*** create new job to recreate all pkg pages
+*** create json
+** misc:
+*** write into all the stats_ tables
+*** make build.sh respect pacman exit code
+*** comparing versions (in scheduler) is probably needed (though it wuld be better if not): if its not empty it must be higher (due to repo constraints), but it can be even higher than in the repo, because we build trunk
+*** debian uses a table removed_packages, should this be used here too?
+*** debian uses a table manual_scheduler to limit the amount of packages someone can schedule per day...
+*** should breakages job look for archlinux breakages? for which cases? a.) grep "was killed after running into timeout after 30m" pkg.html
+** using notes.git should be next!
+
+* rebuilding against the archlinux repos
+** only needs to be done once using https://github.com/Foxboron/devtools-repro
+** to not (pacman know about trunk and thus) build trunk, we need to configure asp.
+** we'll keep building against repo+trunk as we do now (so that archlinux can also benefit from the QA effects)
+
+* fix build.sh:
+** build2.log doesn't get deleted if build1 fails
+** -> rename build2.log to $version_build2.log (don't include package name...)
+
+* things to be done before enabling more builders:
+** build in /srv/workspace instead of /tmp (once this has been done reduce /tmp size back to 15G)
+
+* build on OSUOSL nodes:
+** update thanks & readme
+
+==== reproducible qubes
+
+* add qubes test on t.r-b.o
+----
+ git clone https://github.com/qubesos/qubes-builder
+ make get-sources BUILDERCONF=scripts/travis-builder.conf COMPONENTS=installer-qubes-os
+ export DIST_DOM0=fc23
+ export USE_QUBES_REPO_VERSION=3.2
+ export INSTALLER_KICKSTART=/tmp/qubes-installer/conf/travis-iso.ks
+
+ make qubes iso BUILDERCONF=scripts/travis-builder.conf VERBOSE=0 COMPONENTS=installer-qubes-os
+----
+* depends: apt install createrepo python-yaml
+* once this iso is being tested, it will be interesting to build the Qubes templates as well, as those images (Qubes templates are images) will be copied on the installation iso. the above iso is a stripped down iso without templates… (and not the real thing)
+
+==== reproducible guix
+
+* there's no "apt-get install", because of non-FHS conformance, but see https://www.gnu.org/software/guix/download/
+* there's a privileged build daemon, which is needed to perform fully isolated builds, see https://www.gnu.org/software/guix/manual/html_node/Build-Environment-Setup.html#Build-Environment-Setup
+* it's a bit of work to set up, but all the steps are documented. the "binary installation" method being the easiest.
+* Manolis wrote:
+----
+There are two ways to install guix, through prebuilt binaries or through
+the source.
+
+*Binary installation:
+
+Go to
+<http://www.gnu.org/software/guix/manual/html_node/Binary-Installation.html>,
+grab the tarball and follow the instructions there.
+
+*Source instalation:
+
+First make sure you have the dependencies mentioned at
+<http://www.gnu.org/software/guix/manual/guix.html#Requirements> installed.
+
+Then download Guix's source from
+ftp://alpha.gnu.org/gnu/guix/guix-0.9.0.tar.gz and use the usual
+./configure && make && make install
+
+After you have Guix built, you need to create the build-users and have
+the guix-daemon run as root, as described here
+<https://www.gnu.org/software/guix/manual/html_node/Build-Environment-Setup.html>.
+
+Keep in mind that the guix-daemon must always run as root.
+
+*Testing if everything works:
+
+Now just run `guix package -i vim` as a non-root user. If it runs
+correctly, Guix is ready for work.
+----
+
+
+=== qa.debian.org*
+
+* udd-versionskew: explain jobs in README
+* udd-versionskew: also provide arch-relative version numbers in output too
+
+=== d-i_manual*
+
+* d-i_check_jobs.sh: check for removed manuals (but with existing jobs) missing
+* svn:trunk/manual/po triggers the full build, should trigger language specific builds.
+* svn:trunk/manual is all thats needed, not whole svn:trunk
+
+=== d-i_build*
+
+* d-i_check_jobs.sh: check for removed package (but with existing jobs) missing
+* build packages using jenkins-debian-glue and not with the custom scripts used today?
+* run scripts/digress/ ?
+
+=== chroot-installation_*
+
+* use schroot for chroot-installation, stop using plain chroot everywhere
+
+== Debian Packaging related
+
+This setup should ideally come as a Debian source package, one far away day maybe.
+
+
+// vim: set filetype=asciidoc:
=====================================
update_jdn.sh
=====================================
@@ -6,24 +6,14 @@
# puppet / salt / ansible / fai / chef / deployme.app - disclaimer
# (IOW: this script has been grown in almost 500 commits and it shows…)
-#
# yes, we know… and: "it" should probably still be done.
-#
# It just unclear, how/what, and what we have actually mostly works.
#
-# Switching to jenkins.debian.org is probably an opportunity
-# to write (refactor this into) *yet another deployment script*
-# (interacting with the DSA machine setup which is in puppet…),
-# thus obsoleting this script gradually, though this is used on
-# 47 hosts currently (of which quite some were initially installed
-# manually…)
-#
-# so, yes, patches welcome. saying this is crap alone is not helpful,
-# nor is just suggesting some new or old technology. patches most welcome!
-#
-# that said, there's a new one: init_node ;)
+# on new nodes first run: init_node
+# and then, don't call this script directly, but rather use bin/djm
set -e
+set -o pipefail # see eg http://petereisentraut.blogspot.com/2010/11/pipefail.html
# prevent failure on first run...
[ ! -f /srv/jenkins/bin/common-functions.sh ] || . /srv/jenkins/bin/common-functions.sh
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/compare/f8e34effb4a4e7e3f378ddc8999b9bd2df4fdd8b...99ff468ee70046d842bdbd14832fb4b96db08aca
--
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/compare/f8e34effb4a4e7e3f378ddc8999b9bd2df4fdd8b...99ff468ee70046d842bdbd14832fb4b96db08aca
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20230617/1e056aea/attachment-0001.htm>
More information about the Qa-jenkins-scm
mailing list