[Git][qa/jenkins.debian.net][master] rdn: dont keep all db backups forever (save like 1tb diskspace)

[Holger Levsen (@holger)]

Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net


Commits:
32de0b70 by Holger Levsen at 2026-06-09T23:50:30+02:00
rdn: dont keep all db backups forever (save like 1tb diskspace)

Signed-off-by: Holger Levsen <holger at layer-acht.org>

- - - - -


2 changed files:

- bin/rebuilderd_db_backup.sh
- bin/rebuilderd_db_backup_compress.sh


Changes:

=====================================
bin/rebuilderd_db_backup.sh
=====================================
@@ -4,7 +4,8 @@
 # released under the GPLv2
 
 #
-# backup a rebuilderd instance - this is rather clumsy but best we can do until rebuilderd uses postgresql or some such
+# backup a rebuilderd instance - this is rather clumsy but best we can do until rebuilderd uses postgresql
+# 	or at least https://github.com/kpcyrd/rebuilderd/issues/244 has been addressed
 #
 # rbuilderd should not be running when this is run to ensure db consistancy.
 # this is enforced by the service running this


=====================================
bin/rebuilderd_db_backup_compress.sh
=====================================
@@ -1,17 +1,35 @@
-#!/bin/bash
-#
-# Copyright 2025 Holger Levsen (holger at layer-acht.org)
+
+# Copyright 2025-2026 Holger Levsen (holger at layer-acht.org)
 # released under the GPLv2
 
 #
-# compress all rebuilderd backups from today
+# compress rebuilderd backups and delete old ones except one per month
 #
 set -e
 
+#
 # actually compress those backups
+#
 TODAY=$(date '+%Y-%m-%d' -u)
 FILES=$(find /srv/rebuilderd/*/stats/db-backup/db-back-$TODAY -type f ! -name '*.zst' ! -name '*.xz' 2>/dev/null || exit 0)
 for i in $FILES ; do
 	zstdmt --rm -4 $i
 done
 
+#
+# actually delete old backups
+#
+THIS_MONTH=$(date -u '+%Y-%m')
+LAST_MONTH=$(date -u -d '1 month ago' '+%Y-%m')
+LAST_2_MONTHS=$(date -u -d '2 months ago' '+%Y-%m')
+# loop through all archs
+for DIR in /srv/rebuilderd/*/stats/db-backup/ ; do
+	cd $DIR
+	# loop through all months except the last two months
+	for MONTH_DIRECTORY in $(find db-back-* -type d | cut -d '-' -f1-4 | sort -u | grep -v -E "($THIS_MONTH|$LAST_MONTH|$LAST_2_MONTHS)") ; do
+		# keep the last backup of each month
+		KEEP=$(find ${MONTH_DIRECTORY}-* -type d | sort -u | tail -1)
+		# delete the rest
+		find ${MONTH_DIRECTORY}-* -name "*.zst" -type f | grep -v $KEEP | xargs -r rm
+	done
+done



View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/32de0b70b0c1e2c4f5f5add87cc38fe89807e354

-- 
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/32de0b70b0c1e2c4f5f5add87cc38fe89807e354
You're receiving this email because of your account on salsa.debian.org. Manage all notifications: https://salsa.debian.org/-/profile/notifications | Help: https://salsa.debian.org/help


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20260609/f27efec7/attachment-0001.htm>