[Piuparts-devel] Bug#794575: support testing foreign arch installation

[Helmut Grohne]

Package: piuparts
Version: 0.64
Severity: wishlist
Tags: patch
User: helmutg at debian.org
Usertags: rebootstrap

During a partial archive cross rebuild, I discovered that many packages
failed to install. Being faced with many package installation failures
seems strange in the presence of piuparts, but then it occurred to me
that piuparts only tests native installation.

So I tried to test a foreign arch installation of libc6 with piuparts
and noticed that it succeeds despite failing to install the foreign arch
libc6. The initial dpkg --unpack fails, because the architecture is not
added to dpkg, but piuparts swallows that failure. I argue that in this
situation, piuparts should either actually perform the test or fail.

Being interested in actually testing stuff, I looked into extending
piuparts for this use case and I came up with the attached patch. It
makes both

    piuparts --arch amd64 libc6_2.19-19_i386.deb

and
    piuparts --arch amd64 -a libc6:i386

work in a meaningful way. I am not sure whether the new functionality
breaks aspects of piuparts that I did not test. It also is not clear
whether such liberal acceptance of packages and package names is desired
or whether piuparts would want to enforce explicit activation of foreign
arch testing via a new command line flag.

Still I hope that my patch can be a basis for adding this feature.

Helmut
-------------- next part --------------
diff -Nru piuparts-0.64/debian/changelog piuparts-0.64+nmu1/debian/changelog
--- piuparts-0.64/debian/changelog	2015-06-12 13:44:18.000000000 +0200
+++ piuparts-0.64+nmu1/debian/changelog	2015-08-04 16:59:28.000000000 +0200
@@ -1,3 +1,10 @@
+piuparts (0.64+nmu1) UNRELEASED; urgency=medium
+
+  * Non-maintainer upload.
+  * Support testing foreign arch installations. (Closes: #-1)
+
+ -- Helmut Grohne <helmut at subdivi.de>  Tue, 04 Aug 2015 16:59:14 +0200
+
 piuparts (0.64) unstable; urgency=medium
 
   [ Holger Levsen ]
diff -Nru piuparts-0.64/piuparts.py piuparts-0.64+nmu1/piuparts.py
--- piuparts-0.64/piuparts.py	2015-06-12 13:43:13.000000000 +0200
+++ piuparts-0.64+nmu1/piuparts.py	2015-08-04 16:42:46.000000000 +0200
@@ -706,7 +706,7 @@
         os.chmod(self.name, 0o755)
         logging.debug("Created temporary directory %s" % self.name)
 
-    def create(self, temp_tgz=None):
+    def create(self, temp_tgz=None, architectures=()):
         """Create a chroot according to user's wishes."""
         self.panic_handler_id = do_on_panic(self.remove)
         if not settings.schroot:
@@ -728,6 +728,7 @@
         if not settings.schroot:
             self.mount_proc()
             self.mount_selinux()
+        self.add_architectures(architectures)
         self.configure_chroot()
 
         # Copy scripts dirs into the chroot, merging all dirs together,
@@ -892,6 +893,17 @@
                     "\n".join(lines) + "\n")
         logging.debug("sources.list:\n" + indent_string("\n".join(lines)))
 
+    def add_architectures(self, architectures):
+        """Ensure that dpkg accepts the given architectures."""
+        architectures = set(architectures)
+        if architectures:
+            _, native_arch = self.run(["dpkg", "--print-architecture"])
+            architectures.discard(native_arch.strip())
+            _, foreign_arches = self.run(["dpkg", "--print-foreign-architectures"])
+            architectures.difference_update(foreign_arches.splitlines())
+            for arch in architectures:
+                self.run(["dpkg", "--add-architecture", arch])
+
     def enable_testdebs_repo(self, update=True):
         if settings.testdebs_repo:
             if settings.testdebs_repo.startswith("deb"):
@@ -2152,18 +2164,33 @@
         (status, output) = run(["dpkg", "--info", filename])
         p = None
         v = None
+        a = None
         for line in [line.lstrip() for line in output.split("\n")]:
             if line.startswith("Package:"):
                 p = line.split(":", 1)[1].strip()
             if line.startswith("Version:"):
                 v = line.split(":", 1)[1].strip()
+            if line.startswith("Architecture:"):
+                a = line.split(":", 1)[1].strip()
         if p is not None:
+            if a is not None and a != "all":
+                p += ":" + a
             if v is not None:
-                vlist.append(p + "=" + v)
-            else:
-                vlist.append(p)
+                p += "=" + v
+            vlist.append(p)
     return vlist
 
+def collect_architectures(packages):
+    """Collect architecture names from a list of possibly arch qualified
+    package names."""
+    architectures = set()
+    for p in packages:
+        p = p.split("=", 1)[0]
+        p = p.split(":", 1)
+        if len(p) > 1:
+            architectures.add(p[1])
+    return architectures
+
 # Method to process a changes file, returning a list of all the .deb packages
 # from the 'Files' stanza.
 
@@ -2501,9 +2528,10 @@
     os.environ["PIUPARTS_TEST"] = "distupgrade"
 
     packages = unqualify(packages_qualified)
+    architectures = collect_architectures(packages_qualified)
 
     chroot = get_chroot()
-    chroot.create()
+    chroot.create(architectures=architectures)
 
     if settings.end_meta:
         # load root_info and selections
@@ -2537,9 +2565,9 @@
 
         chroot = get_chroot()
         if temp_tgz is None:
-            chroot.create()
+            chroot.create(architectures=architectures)
         else:
-            chroot.create(temp_tgz)
+            chroot.create(temp_tgz, architectures=architectures)
             chroot.remove_temp_tgz_file(temp_tgz)
             dont_do_on_panic(panic_handler_id)
 
@@ -3054,9 +3082,11 @@
         packages = package_list
         package_files = []
 
+    architectures = collect_architectures(packages)
+
     if len(settings.debian_distros) == 1:
         chroot = get_chroot()
-        chroot.create()
+        chroot.create(architectures=architectures)
 
         chroot_state = {}
         chroot_state["tree"] = chroot.save_meta_data()