[Piuparts-commits] rev 921 - piatti/home/piupartss trunk trunk/debian
Holger Levsen
holger at alioth.debian.org
Sun Jul 10 13:11:19 UTC 2011
Author: holger
Date: 2011-07-10 13:11:19 +0000 (Sun, 10 Jul 2011)
New Revision: 921
Modified:
piatti/home/piupartss/crontab
trunk/TODO
trunk/debian/changelog
trunk/piuparts-slave.py
Log:
* piuparts-slave:
- check if chroot-tgz is older than MAX_TGZ_AGE (currently
hardcoded to 30) and recreate it, if it is. Keep backup and put in back
in place when debootstrapping a new chroot-tgz fails. Thanks to Scott
Schaefer for the patch. (Closes: #632924)
- fix bug in piuparts-slave which prevented running a single section by
including section name as command-line argument - thanks again to Scott
and see 632924 too.
piatti/home/piupartss/crontab: disable deleting of chroot-tgzs
Modified: piatti/home/piupartss/crontab
===================================================================
--- piatti/home/piupartss/crontab 2011-07-10 12:58:04 UTC (rev 920)
+++ piatti/home/piupartss/crontab 2011-07-10 13:11:19 UTC (rev 921)
@@ -14,8 +14,3 @@
#
23 */6 * * * /home/piupartss/bin/detect_leftover_processes
-#
-# delete chroot-tarballs once a month, so that new ones are created automatically
-# after that, kill and restart the slave session to recreate them
-#
-42 23 1 * * sudo rm -v /org/piuparts.debian.org/slave/*/*.tar.gz ; sudo screen -r piuparts-slave-screen -X quit ; sleep 60 ; sudo /home/piupartss/bin/slave_run
Modified: trunk/TODO
===================================================================
--- trunk/TODO 2011-07-10 12:58:04 UTC (rev 920)
+++ trunk/TODO 2011-07-10 13:11:19 UTC (rev 921)
@@ -4,9 +4,12 @@
Please also check the BTS, especially the bugs with a severity
higher than wishlist!
+for 0.41
-for 0.41:
+- piuparts-slave: make MAX_TGZ_AGE configurable
+for 0.42:
+
- piuparts-report should keep state, ie cache the results of the log
analysis and generated html pages. Thus, a piuparts-report run could
take minutes instead of hours... this is now (with four distros tested
@@ -21,7 +24,7 @@
- fix scripts for dash as default shell
-for 0.42:
+for 0.43:
- use python-debianbts to track bugginess status of packages via
the bts and remove the "bugged" directory-status tracking
@@ -38,7 +41,7 @@
why and fix it.
-for 0.43:
+for 0.44:
- provide mail templates for bugreports (which ideally only
should need manual logfile trimming) at
@@ -57,26 +60,19 @@
- publish FAI classes to setup piuparts.$fqdn automatically.
-for 0.44:
+for 0.45:
- PackagesDB: get rid of 50% of unique()
- remove configfile option "mirror" and use the mirror from
"packages-url".
-- piuparts-slave: if chroot.tgz is older than N days, regenerate
- it
- - currently this is done via ~piupartss/crontab, but that's a
- hack
- - only throw away tarballs (and recreate them) after testing
- that they can be recreated
-
- master:
- write stats about number of daily tests (-> report: create
fancy graphs)
-for 0.45:
+for 0.46:
- slave_run: cleanup stale proc mountpoints
@@ -90,7 +86,7 @@
prevent start-stop-daemon from starting any daemons ;)
-for 0.46 and later:
+for 0.47 and later:
- report:
- write stats about the reasons for failures, as its done with
Modified: trunk/debian/changelog
===================================================================
--- trunk/debian/changelog 2011-07-10 12:58:04 UTC (rev 920)
+++ trunk/debian/changelog 2011-07-10 13:11:19 UTC (rev 921)
@@ -28,12 +28,20 @@
- Thanks to Scott Schaefer for the patch. (Closes: #632046)
- new option "--no-install-purge-test" to only do upgrade tests
- Thanks to Andreas Bergmann for the patch (Closes: #588482)
+ * piuparts-slave:
+ - check if chroot-tgz is older than MAX_TGZ_AGE (currently
+ hardcoded to 30) and recreate it, if it is. Keep backup and put in back
+ in place when debootstrapping a new chroot-tgz fails. Thanks to Scott
+ Schaefer for the patch. (Closes: #632924)
+ - fix bug in piuparts-slave which prevented running a single section by
+ including section name as command-line argument - thanks again to Scott
+ and see 632924 too.
* debian/control:
- require python >= 2.6.6-3~, get rid of python-support build-depends.
- add X-Python-Version: pseudo-header
* debian/rules: use dh_python2 instead of python-support to build the
package
- * Makefile: build for python 2.6 and 2.7 instead of 2.5 and 2.6
+ * Makefile: build for python 2.6 and 2.7 instead of 2.5 and 2.6
-- Holger Levsen <holger at debian.org> Sat, 25 Jun 2011 23:33:49 +0200
Modified: trunk/piuparts-slave.py
===================================================================
--- trunk/piuparts-slave.py 2011-07-10 12:58:04 UTC (rev 920)
+++ trunk/piuparts-slave.py 2011-07-10 13:11:19 UTC (rev 921)
@@ -25,17 +25,17 @@
import os
import sys
+import stat
import time
import logging
import ConfigParser
-import urllib
-
import piupartslib.conf
import piupartslib.packagesdb
CONFIG_FILE = "/etc/piuparts/piuparts.conf"
+MAX_TGZ_AGE = 60*60*24*30
def setup_logging(log_level, log_file_name):
@@ -235,13 +235,15 @@
if self._config["chroot-tgz"] and not self._config["distro"]:
logging.info("The option --chroot-tgz needs --distro.")
- if self._config["chroot-tgz"] and not os.path.exists(self._config["chroot-tgz"]):
- create_chroot(self._config, self._config["chroot-tgz"], self._config["distro"])
+ tarball = self._config["chroot-tgz"]
+ if tarball:
+ create_or_replace_chroot_tgz(self._config, tarball,
+ MAX_TGZ_AGE, self._config["distro"])
- if (self._config["upgrade-test-distros"] and self._config["upgrade-test-chroot-tgz"]
- and not os.path.exists(self._config["upgrade-test-chroot-tgz"])):
- create_chroot(self._config, self._config["upgrade-test-chroot-tgz"],
- self._config["upgrade-test-distros"].split()[0])
+ tarball = self._config["upgrade-test-chroot-tgz"]
+ if self._config["upgrade-test-distros"] and tarball:
+ create_or_replace_chroot_tgz(self._config, tarball,
+ MAX_TGZ_AGE, self._config["upgrade-test-distros"].split()[0])
for rdir in ["new", "pass", "fail"]:
rdir = os.path.join(self._slave_directory, rdir)
@@ -416,6 +418,21 @@
f.close()
os.rename(tarball + ".new", tarball)
+def create_or_replace_chroot_tgz(config, tgz, max_age, distro):
+ age = 0
+ if os.path.exists(tgz):
+ age = time.time() - os.stat(tgz)[stat.ST_CTIME]
+ if age > max_age:
+ os.rename(tgz, tgz + ".old")
+ logging.info("%s too old. Renaming to force re-creation" % tgz)
+ if not os.path.exists(tgz):
+ create_chroot(config, tgz, distro)
+ if not os.path.exists(tgz) and age > 0 and age > max_age:
+ os.rename(tgz + ".old", tgz)
+ logging.info("Failed to create ... reverting to old %s" % tgz)
+ create_chroot(config, tgz, distro)
+ else:
+ os.unlink(tgz + ".old")
def fetch_packages_file(config, distro):
mirror = config["mirror"]
@@ -448,11 +465,11 @@
# argument referring to a section in configuration file.
# If no argument is given, the "global" section is assumed.
section_names = []
+ global_config = Config(section="global")
+ global_config.read(CONFIG_FILE)
if len(sys.argv) > 1:
section_names = sys.argv[1:]
else:
- global_config = Config(section="global")
- global_config.read(CONFIG_FILE)
section_names = global_config["sections"].split()
sections = []
More information about the Piuparts-commits
mailing list