[Piuparts-commits] [piuparts] 04/07: p-s: use the --save-end-meta and --end-meta options

[Holger Levsen]

This is an automated email from the git hooks/post-receive script.

holger pushed a commit to branch develop
in repository piuparts.

commit 0e6cd31438d99bf3e4044e07d211a95759b90013
Author: Andreas Beckmann <anbe at debian.org>
Date:   Thu Mar 16 23:05:20 2017 +0100

    p-s: use the --save-end-meta and --end-meta options
    
    should speed up distupgrade tests
    
    each slave instance maintains a separate state file for each config
    section (distupgrade tests only)
    cached chroot state information is valid for 6 hours (unless a
    mismatch in avail_md5_history is found earlier)
    
    Signed-off-by: Andreas Beckmann <anbe at debian.org>
    Signed-off-by: Holger Levsen <holger at layer-acht.org>
---
 README_server.txt |  9 +++++++++
 debian/changelog  |  3 +++
 piuparts-slave.py | 22 ++++++++++++++++++++++
 3 files changed, 34 insertions(+)

diff --git a/README_server.txt b/README_server.txt
index ab983e7..3395631 100644
--- a/README_server.txt
+++ b/README_server.txt
@@ -443,6 +443,15 @@ section, too, and will serve as defaults for all other sections
  automatically selected default name) is located. The default is
  '.'.
 
+* "chroot-meta-auto" (global, section) is a file in the section
+ directory where the slave will store cached chroot meta data for
+ the reference target chroot in distupgrade tests. This speeds up
+ distupgrade tests since it avoids doing an empty upgrade test to
+ generate this data on-the-fly as part of each test. Cached data
+ will be valid for 6 hours unless a mismatch in the package
+ versions available in the chroot is detected earlier.
+ This is not set (and therefore not enabled) by default.
+
 * "upgrade-test-distros" is the space delimited list of
  distributions the slave should use for testing upgrades
  between distributions (i.e., Debian versions). Using "partial"
diff --git a/debian/changelog b/debian/changelog
index 1b61952..affaaa0 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -17,6 +17,9 @@ piuparts (0.77) UNRELEASED; urgency=medium
   * piuparts-slave.py:
     - Run piuparts on dummy package "TARBALL" (but skip tests) to create
       tarballs.
+    - Add "chroot-meta-auto" config option to automatically maintain cached
+      target chroot meta data (using --end-meta and --save-end-meta) for
+      distupgrade tests instead of generating it on-the-fly.
   * piuparts-analyze.py:
   * piuparts-report.py:
     - Plot all states in the graphs.
diff --git a/piuparts-slave.py b/piuparts-slave.py
index cace98d..5b178f3 100644
--- a/piuparts-slave.py
+++ b/piuparts-slave.py
@@ -95,6 +95,7 @@ class Config(piupartslib.conf.Config):
                                          "chroot-tgz": None,
                                          "upgrade-test-distros": None,
                                          "basetgz-directory": ".",
+                                         "chroot-meta-auto": None,
                                          "max-reserved": 1,
                                          "debug": "no",
                                          "keep-sources-list": "no",
@@ -640,6 +641,15 @@ class Section:
         self._check_tarball()
         if not os.path.exists(self._get_tarball()):
             self._error_wait_until = time.time() + 300
+        if self._config["chroot-meta-auto"]:
+            if os.path.exists(self._config["chroot-meta-auto"]):
+                try:
+                    age = time.time() - os.path.getmtime(self._config["chroot-meta-auto"])
+                    if age > 6 * 3600:
+                        os.unlink(self._config["chroot-meta-auto"])
+                        logging.info("Deleting old %s" % self._config["chroot-meta-auto"])
+                except OSError:
+                    pass
         for package_name, version in self._slave.get_reserved():
             self._throttle_if_overloaded()
             if interrupted or got_sighup:
@@ -690,6 +700,11 @@ class Section:
                 command.extend(["-d", distro])
         if self._config["keep-sources-list"] in ["yes", "true"]:
             command.append("--keep-sources-list")
+        if distupgrade and self._config["chroot-meta-auto"]:
+            if not os.path.exists(self._config["chroot-meta-auto"]):
+                command.extend(["-S", self._config["chroot-meta-auto"]])
+            else:
+                command.extend(["-B", self._config["chroot-meta-auto"]])
         command.extend(["--apt", "%s=%s" % (pname, pvers)])
 
         subdir = "fail"
@@ -766,6 +781,13 @@ class Section:
             elif not "piuparts run ends" in lastline:
                 ret += 1024
                 output.write(" *** PIUPARTS OUTPUT INCOMPLETE ***\n")
+            elif distupgrade and self._config["chroot-meta-auto"] and \
+                    "History of available packages does not match - reference chroot may be outdated" in f:
+                try:
+                    os.unlink(self._config["chroot-meta-auto"])
+                    logging.info("Deleting outdated %s" % self._config["chroot-meta-auto"])
+                except OSError:
+                    pass
 
         output.write("\n")
         output.write("ret=%d\n" % ret)

-- 
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/piuparts/piuparts.git