[Piuparts-devel] Bug#1055508: piuparts: helmut's little wishlist

[Helmut Grohne]

Package: piuparts
Version: 1.2
Severity: wishlist

Hi Holger and Nicolas,

I've got some minor itches with piuparts and would like to figure out
which of them you are interested addressing such that I can file
corresponding MRs on salsa. In case you don't find this useful, please
just close the bug.

# tar compression

When using piuparts with --basetgz it really only accepts something
compressed with gzip. Other tools (schroot, pbuilder, etc.) support
other compression schemes and zstd is a popular contender, because it
can beat gzip on both ratio and speed at the same time.

piuparts has one "tar" invocation for performing the decompression and
passes the "-z" flag there. If we were to change that flag to
"--auto-compress", piuparts would immediately support all sorts of
compression formats. Just the option name "--basetgz" could become a
little misleading. Is that something worth exploring in a MR?

@@ ... @@ unpack_from_tgz
-run(prefix + ["tar", "-C", self.name, "-zxf", tarball])
+run(prefix + ["tar", "-C", self.name, "--auto-compress", "-xf", tarball])

# support mmdebstrap as a debootstrap alternative

If using piuparts without providing some chroot, it'll create one using
debootstrap. Since debootstrap is very compatible, it is relatively slow
compared to mmdebstrap. At the same time, the target audience of
piuparts is Debian, so mmdebstrap will be readily available. Also
mmdebstrap has an extensive test suite that validates its output to be
the same as debootstrap. Since piuparts sets up the target, one has to
pass --skip=check/empty to mmdebstrap though. Is that worth adding
another option?

# mount_proc

When mount_proc notices the absence of /dev/ptmx, it'll create a device
node there and the bind mounts /dev/pts/ptmx there. If running in a
container, mknod may fail and a non-recursive bind mount may fail as
well. A simple variant achieving similar semantics is creating /dev/ptmx
as a symlink when it is entirely missing. If anything, having one less
mount provides a small speedup. Do you want a MR?

@@ ... @@ mount_proc
         dev_ptmx_rel_path = self.relative("dev/ptmx")
         if not os.path.islink(dev_ptmx_rel_path):
             if not os.path.exists(dev_ptmx_rel_path):
-                os.mknod(dev_ptmx_rel_path, 0o0666 | stat.S_IFCHR, os.makedev(5, 2))
-            self.mount(self.relative("dev/pts/ptmx"), "/dev/ptmx", opts="bind", no_mkdir=True)
+                os.symlink("pts/ptmx", dev_ptmx_rel_path)
+            else:
+                self.mount(self.relative("dev/pts/ptmx"), "/dev/ptmx", opts="bind", no_mkdir=True)
         p = subprocess.Popen(["tty"], stdout=subprocess.PIPE,
                              universal_newlines=True)

# umount_all

In some of my experiments, I've seen umount_all fail. Some umount failed
to work and when it panic()ed, it failed on an infinite recursion. I
happen to not have understood the details. I just noticed that adding
--lazy to umount would fix this very problem. It enables you to umount
busy mount points reliably by hiding them from everything else and
letting the kernel garbage collect them when the last user exits. Do you
want that?

@@ ... @@ umount_all
-run(["umount", mountpoint], ignore_errors=True)
+run(["umount", "--lazy", mountpoint], ignore_errors=True)

# Default to recursive bind mounts

When specifying --bind, piuparts performs a non-recursive bind mount.
That may be unintuitive and it also is prohibited in containers. Would
you mind changing bind mounts to recursive ones in general? Is there a
need to support non-recursive ones at all?

@@ ... @@ configure_chroot
 for bindmount in settings.bindmounts:
-    self.mount(bindmount, bindmount, opts="bind")
+    self.mount(bindmount, bindmount, opts="rbind")

# Allow bind mounting non-directories

When specifying --bind, piuparts implies that the thing you mount is a
directory. It could be a device or a file however. When trying to do
that, piuparts still creates a target directory and fails the mount
operation as you cannot bind mount a non-directory on a directory. It
could easily check this situation and create a non-directory node in
such situations. Do you want a MR?

@@ ... @@ mount
 path = canonicalize_path(self.name, path)
+fullpath = self.relative(path)
 if not no_mkdir:
-    self.mkdir_p(path)
-fullpath = self.relative(path)
+    is_non_bind = not set(("bind", "rbind")).intersection((opts or "").split(","))
+    if is_non_bind or os.path.isdir(source):
+        self.mkdir_p(path)
+    elif not os.path.exists(fullpath):
+        self.mkdir_p(os.path.dirname(path))
+        os.mknod(fullpath, stat.S_IFREG)


Thank you for considering

Helmut