[Piuparts-devel] [Git][debian/piuparts][develop] WIP: reduce number of mails per day, from 3 down to 2, see #912591.

[Holger Levsen]

Holger Levsen pushed to branch develop at Debian / piuparts


Commits:
107b4297 by Holger Levsen at 2018-11-29T10:47:05Z
WIP: reduce number of mails per day, from 3 down to 2, see #912591.

master-bin/generate_daily_report:
- redirect logs of called scripts to htdocs/logs.
- include listing of htdocs/logs/$TODAY into daily report.
- publish dailyreport.txt in htdocs/logs/$TODAY instead of htdocs/.

Signed-off-by: Holger Levsen <holger at layer-acht.org>

- - - - -


2 changed files:

- debian/changelog
- master-bin/generate_daily_report.in


Changes:

=====================================
debian/changelog
=====================================
@@ -7,6 +7,10 @@ piuparts (0.95) UNRELEASED; urgency=medium
     - lib/write_log.sh: new helper to publish logs in htdocs/logs.
     - master-bin/master_shell_runner: new script to run python scripts to
       redirect logs to htdocs/logs.
+    - master-bin/generate_daily_report:
+      - redirect logs of called scripts to htdocs/logs.
+      - include listing of htdocs/logs/$TODAY into daily report.
+      - publish dailyreport.txt in htdocs/logs/$TODAY instead of htdocs/.
     - master-bin/detect_archive_issues, detect_piuparts_issues,
       detect_network_issues, reschedule_oldest_logs, rotate_master_logs and
       report_untestable_packages: redirect logs to htdocs/logs.


=====================================
master-bin/generate_daily_report.in
=====================================
@@ -1,6 +1,7 @@
 #!/bin/sh
 
 # Copyright © 2011-2013 Andreas Beckmann (anbe at debian.org)
+# Copyright © 2018 Holger Levsen (holger at layer-acht.org)
 #
 # This program is free software; you can redistribute it and/or modify it
 # under the terms of the GNU General Public License as published by the
@@ -15,8 +16,8 @@
 # You should have received a copy of the GNU General Public License along
 # with this program. If not, see <https://www.gnu.org/licenses/>
 
-
 . @sharedir@/piuparts/lib/read_config.sh
+. @sharedir@/piuparts/lib/write_log.sh
 
 get_config_value MASTER global master-directory
 get_config_value SECTIONS global sections
@@ -24,7 +25,6 @@ get_config_value HTDOCS global output-directory
 get_config_value URLBASE global urlbase https://piuparts.debian.org
 get_config_value PYTHONPATH global PYTHONPATH ''
 
-
 export PYTHONPATH
 
 LOCKFILE=$HTDOCS/daily.lock
@@ -36,22 +36,26 @@ fi
 date > $LOCKFILE
 
 R_STARTDATE=$(date -u +%s)
+R_LOGDATEDIR="$HTDOCS/logs/$(date -u '+%Y/%m/%d')"
 
 #
-# various scripts to detect issues and cleanup
+# various scripts to detect issues and cleanup to be run right before we generate the daily report
+# (this code block contains cruft and could be refactored)
 #
 OUTPUT=$(mktemp)
+LOG_OUTPUT=$(mktemp)
 for SCRIPT in archive_old_logs ; do
   STARTDATE=$(date -u +%s)
   @sharedir@/piuparts/master/$SCRIPT 2>&1 > $OUTPUT
   FINALDATE=$(date -u +%s)
   if [ -s $OUTPUT ] ; then
 	if grep -q "Runtime: " $OUTPUT ; then
-		cat $OUTPUT | mail -s $SCRIPT $LOGNAME
+		cat $OUTPUT > $LOG_OUTPUT
 	else
 		RUNTIME=$(date -u -d "0 $FINALDATE seconds - $STARTDATE seconds" +%T)
-		( echo "Runtime: $RUNTIME" ; echo ; cat $OUTPUT ) | mail -s $SCRIPT $LOGNAME
+		( echo "Runtime: $RUNTIME" ; echo ; cat $OUTPUT ) > $LOG_OUTPUT
 	fi
+	publish_logs $LOG_OUTPUT "" $HTDOCS $SCRIPT
   fi
 done
 rm $OUTPUT
@@ -96,7 +100,14 @@ rm $expire
 FINALDATE=$(date -u +%s)
 RUNTIME=$(date -u -d "0 $FINALDATE seconds - $R_STARTDATE seconds" +%T)
 
-( echo "Runtime: $RUNTIME" ; echo ; cat $DAILYREPORT ) | mail -s generate_daily_report $LOGNAME
+(
+	echo "Runtime: $RUNTIME"
+	echo
+	ls -lart $R_LOGDATEDIR
+	echo
+	cat $DAILYREPORT
+) | mail -s generate_daily_report $LOGNAME
+mv $DAILYREPORT $R_LOGDATEDIR
 
 mv $FAILURESTAMP.new $FAILURESTAMP
 rm -f $LOCKFILE



View it on GitLab: https://salsa.debian.org/debian/piuparts/commit/107b4297849027366c06d27cec31d47ae2c071b2

-- 
View it on GitLab: https://salsa.debian.org/debian/piuparts/commit/107b4297849027366c06d27cec31d47ae2c071b2
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/piuparts-devel/attachments/20181129/9a6c4565/attachment-0001.html>