[Qa-jenkins-scm] [Git][qa/jenkins.debian.net][master] djm: automatically split out previous months data from logfiles into $logfile.YYYY-MM

[Holger Levsen (@holger)]

Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net


Commits:
2e384f30 by Holger Levsen at 2023-06-01T21:49:04+02:00
djm: automatically split out previous months data from logfiles into $logfile.YYYY-MM

Signed-off-by: Holger Levsen <holger at layer-acht.org>

- - - - -


2 changed files:

- TODO
- bin/djm


Changes:

=====================================
TODO
=====================================
@@ -32,22 +32,22 @@ See link:https://jenkins.debian.net/userContent/about.html["about jenkins.debian
 ** maybe: rm /tmp/mmdebstrap.* older than 3 days
 * split TODO in TODO and TODO.legacy?
 * djm:
+** djm-jenkins-parser needs to split out old data too, maybe run parser with triggering option after having done this locally?
+** rename .djm-jenkins-parser.log to something with .raw
+** rename .djm-jenkins-ui.log to reflect its from .raw (but now has dates)
+** cleanup parser-log too, eg if its too big. then also figlet a warning...
+** new feature: --show-month 04
+** --report: include hours with manual jobs triggered
 ** action: rk / remove-oldest-kernel
 ** action: sm / shell-monitor
 ** action: rj / restart-jenkins
 ** action: mo / mark-offline
 ** action: so / show-offline
 ** action: ao / all-online / mark all online
-** make sure there's only one djm run fetching...
-** action: s|top-builds: prepare reboot on jenkins: 1st stop build service, 2nd stop builds on all nodes
-** automatically backup+cleanup logs on the first of the month
-*** write it now, to move April data to 04 files, and either make this run on May 23 or just any day?
-** new feature: --show-month 04
-** --report: include hours with manual jobs triggered
+** action: s|top-builds: 1st stop build service, 2nd stop builds on all nodes
 ** option: --today to be used with action shell (and maybe others)
 ** option: --dont-wait4-enter, default being wait for enter.
-** update documentation refering to deploy_jdn to djm
-*** ups, there's no doc on deploy_jdn anyway
+** write documentation for djm
 ** also log an id, so that one command effecting several hosts is counted as on action
 *** make actions triggering 'all' be only one entry in the djm logfile -> more sensible stats
 ** maybe:
@@ -55,8 +55,6 @@ See link:https://jenkins.debian.net/userContent/about.html["about jenkins.debian
 *** document jenkins plugin updates -> manually or though my local script which calls the webpages could log this more easily
 *** include notmuch mails, "received and read". write djm-notmuch-parser.
 *** include irc. (timeslices when one said something on relevant channels)
-*** new action to stop the build service on jenkins and all nodes
-*** new action to remove old kernels on old codethink/ubuntu nodes
 ** remove "so expect changes for some time." line
 ** remove/reword "Still very simple statistics"
 


=====================================
bin/djm
=====================================
@@ -264,13 +264,34 @@ djm_fetch() {
 	#
 	# parse jenkins build logs and copy the result over
 	#
-	LOCK=${LOCAL_LOGFILE}.lock
+	local LOCK=${LOCAL_LOGFILE}.lock
 	if [ ! -f ${LOCK} ] ; then
 		touch $LOCK
+		# run the parsers
 		ssh $DJM_USER at jenkins.debian.net "DJM_USER=$DJM_USER /srv/jenkins/bin/djm-jenkins-parser"
+		# fetch
 		scp $DJM_USER at jenkins.debian.net:$(basename $UI_LOGFILE) $UI_LOGFILE
 		scp $DJM_USER at jenkins.debian.net:$(basename $PARSER_LOGFILE) $PARSER_LOGFILE
 		ssh $DJM_USER at jenkins.debian.net "cd ~jenkins/jobs ; ls -1d reproducible_* |wc -l" > $JOBS
+		# split old data out
+		if grep -q -v ^$LOGMONTH $LOGFILE || grep -q -v ^$LOGMONTH $UI_LOGFILE ; then
+			figlet Splitting out old data...
+			local file
+			local month
+			local tmpfile=$(mktemp)
+			for file in $LOGFILE $UI_LOGFILE ; do
+				for month in $(grep -v ^$LOGMONTH $file | cut -d '-' -f1-2 | sort -u) ; do
+					echo "Moving old entries from $month in $file to $file.$month."
+					grep ^$month $file >> $file.$month
+					mv $file.$month $tmpfile
+					sort -u $tmpfile > $file.$month
+					grep -v ^$month $file > $tmpfile
+					sort -u $tmpfile > $file
+					rm $tmpfile
+				done
+			done
+			rm -f $tmpfile
+		fi
 		rm $LOCK
 	else
 		echo "Warning: $LOCK exists, not fetching remote log files."
@@ -596,7 +617,17 @@ djm_do() {
 		#
 		# log
 		#
-		echo "$LOGDATE, $SHORTNODE, $ACTION, $REASON" | tee -a $LOGFILE
+		local LOCK=${LOGFILE}.lock
+		local LOGGED=false
+		while ! $LOGGED ; do
+			if [ ! -f $LOCK ] ; then
+				echo "$LOGDATE, $SHORTNODE, $ACTION, $REASON" | tee -a $LOGFILE
+				LOGGED=true
+			else
+				echo "Cannot log action, as $LOCKFILE exists, waiting 3 secs until retry."
+				sleep 3
+			fi
+		done
 	done
 	
 	echo Thank you for doing documented jenkins maintenance.



View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/2e384f30fa6bd532d539a6c0bf8804426cbd1cb7

-- 
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/2e384f30fa6bd532d539a6c0bf8804426cbd1cb7
You're receiving this email because of your account on salsa.debian.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20230601/f4066e9e/attachment-0001.htm>