Bug#1073922: systemd-{container, cryptsetup, repart}: ineffective Replaces due to /usr-move (DEP17)

[Helmut Grohne]

Control: reassign -1 systemd-container,systemd-cryptsetup,systemd-repart
Control: found -1 systemd/256.1-1
Control: tags -1 + patch

On Thu, Jun 20, 2024 at 10:58:23AM +0200, Helmut Grohne wrote:
> Package: systemd-container,systemd-cryptsetup,cryptsetup-repart

Fixed bad package cryptsetup-repart.

> Let me not go into details of this problem just yet and just install
> this bug as a temporary migration blocker. I shall have an update within
> three working days, ideally with a patch. Thanks for your patience.

The recurring theme is that systemd moved all of its files from / to
/usr (expected via DEP17) and now moves components from the main systemd
package into systemd-container, systemd-cryptsetup and systemd-repart.
In all of these cases, affected files may be lost in upgrades from
either bookworm or bookworm-backports to unstable and eventually trixie.
Users upgrading from trixie to sid, will likely not experience loss
unless they skip systemd versions.

There are multiple mitigation techniques available. Upgrading
Breaks+Replaces to Conflicts provides a relatively strong protection as
long as one uses an apt-based package management tool. However, the CTTE
advised that packages relevant to booting a system should also be safe
when installing packages directly with dpkg and in that scenario,
Conflicts are insufficient, because dpkg allows a conflicting package to
be unpacked before the conflicted package is removed to facilitate a
smooth handover. This is only exercised by apt when the relevant
packages employ a mutual conflict, which is not the case here. As such,
I also add temporary diversions that exist from preinst to postinst to
protect the relevant files from loss.

While I could have just written the maintainer scripts, I expect more
restructuring to happen until the trixie release and hence went for a
templating system. Affected files should be added (with their aliased
path) to debian/$PKG.usrmergemitigate. Then a debian/rules snippet will
construct relevant debian/*.preinst-usrmerge and
debian/*.postinst-usrmerge snippets that substitute
#USRMERGEMITIGATEPREINST# and #USRMERGEMITIGATEPOSTINST# in actual
debian/*.preinst and debian/*.postinst via dh_installdeb's substitution
mechanism. When adding a new debian/*.usrmergemitigate file, one also
has to add these substitutions to the relevant .preinst and .postinst.
I think this bears a good trade-off regarding complexity and repetition.
Let me know whether you disagree with this judgement.

Helmut
-------------- next part --------------
diff --minimal -Nru systemd-256.1/debian/changelog systemd-256.1/debian/changelog
--- systemd-256.1/debian/changelog	2024-06-19 00:19:16.000000000 +0200
+++ systemd-256.1/debian/changelog	2024-06-19 14:51:07.000000000 +0200
@@ -1,3 +1,13 @@
+systemd (256.1-1.1) UNRELEASED; urgency=medium
+
+  * Non-maintainer upload.
+  * Mitigate file loss arising from the combination of restructuring
+    and /usr-move (DEP17 P1, Closes: #-1):
+    + Upgrade relevant Breaks+Replaces to Conflicts.
+    + Add protective diversions for lost files.
+
+ -- Helmut Grohne <helmut at subdivi.de>  Wed, 19 Jun 2024 14:51:07 +0200
+
 systemd (256.1-1) unstable; urgency=medium
 
   * Bump versioned breaks against dracut to 102-2 (Closes: #1073290)
diff --minimal -Nru systemd-256.1/debian/control systemd-256.1/debian/control
--- systemd-256.1/debian/control	2024-06-19 00:19:11.000000000 +0200
+++ systemd-256.1/debian/control	2024-06-19 14:51:07.000000000 +0200
@@ -164,8 +164,7 @@
 Recommends: libnss-mymachines,
             ${dlopen:Recommends},
 Suggests: ${dlopen:Suggests},
-Breaks: systemd (<< 256-2~),
-Replaces: systemd (<< 256-2~),
+Conflicts: systemd (<< 256-2~),
 Description: systemd container/nspawn tools
  This package provides systemd's tools for nspawn and container/VM management:
   * systemd-nspawn
@@ -649,8 +648,7 @@
          ${misc:Depends},
 Recommends: ${dlopen:Recommends},
 Suggests: ${dlopen:Suggests},
-Breaks: systemd (<< 256-2~),
-Replaces: systemd (<< 256-2~),
+Conflicts: systemd (<< 256-2~),
 Description: Provides the systemd-repart utility
  systemd-repart is a configuration-driven system partitioning tool. It follows
  the Discoverable Partitions Specification and provides support for encryption
@@ -663,8 +661,7 @@
          ${misc:Depends},
 Recommends: ${dlopen:Recommends},
 Suggests: ${dlopen:Suggests},
-Breaks: systemd (<< 256-2~),
-Replaces: systemd (<< 256-2~),
+Conflicts: systemd (<< 256-2~),
 Description: Provides cryptsetup, integritysetup and veritysetup utilities
  Utilities and units that integrated cryptsetup/integritysetup/veritysetup
  support in systemd.
diff --minimal -Nru systemd-256.1/debian/postinst.usrmergetemplate systemd-256.1/debian/postinst.usrmergetemplate
--- systemd-256.1/debian/postinst.usrmergetemplate	1970-01-01 01:00:00.000000000 +0100
+++ systemd-256.1/debian/postinst.usrmergetemplate	2024-06-19 14:51:07.000000000 +0200
@@ -0,0 +1,7 @@
+if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ]; then
+	for f in @files@; do
+		dpkg-divert --package #PACKAGE# --no-rename \
+			--divert "/lib/firmware/$f.usr-is-merged" \
+			--remove "/lib/firmware/$f"
+	done
+fi
diff --minimal -Nru systemd-256.1/debian/preinst.usrmergetemplate systemd-256.1/debian/preinst.usrmergetemplate
--- systemd-256.1/debian/preinst.usrmergetemplate	1970-01-01 01:00:00.000000000 +0100
+++ systemd-256.1/debian/preinst.usrmergetemplate	2024-06-19 14:51:07.000000000 +0200
@@ -0,0 +1,7 @@
+if [ "$1" = "upgrade" ] || [ "$1" = "install" ]; then
+	for f in @files@; do
+		dpkg-divert --package #PACKAGE# --no-rename \
+			--divert "/lib/firmware/$f.usr-is-merged" \
+			--add "/lib/firmware/$f"
+	done
+fi
diff --minimal -Nru systemd-256.1/debian/rules systemd-256.1/debian/rules
--- systemd-256.1/debian/rules	2024-06-19 00:19:11.000000000 +0200
+++ systemd-256.1/debian/rules	2024-06-19 14:51:07.000000000 +0200
@@ -244,6 +244,7 @@
 
 execute_after_dh_auto_clean:
 	rm -f debian/shlibs.local
+	rm -f debian/*.preinst-usrmerge debian/*.postinst-usrmerge
 	# remove Python byte code files
 	rm -rf src/ukify/test/__pycache__/
 	rm -rf src/ukify/__pycache__/
@@ -357,5 +358,13 @@
 	env -u LD_PRELOAD meson test -C obj-$(DEB_HOST_GNU_TYPE) --print-errorlogs $(TEST_TIMEOUT_MULTIPLIER)
 endif
 
+debian/%.preinst-usrmerge:debian/%.usrmergemitigate debian/preinst.usrmergetemplate
+	sed "s#@files@#$$(xargs < $<)#;s/#PACKAGE#/$(basename $(notdir $@))/" < debian/preinst.usrmergetemplate > $@
+debian/%.postinst-usrmerge:debian/%.usrmergemitigate debian/postinst.usrmergetemplate
+	sed "s#@files@#$$(xargs < $<)#;s/#PACKAGE#/$(basename $(notdir $@))/" < debian/postinst.usrmergetemplate > $@
+
+override_dh_installdeb:$(foreach p,$(wildcard debian/*.usrmergemitigate),$(basename $(p)).preinst-usrmerge $(basename $(p)).postinst-usrmerge)
+	dh_installdeb $(foreach p,$(wildcard debian/*.usrmergemitigate),-Dpkg.$(basename $(notdir $(p))).USRMERGEMITIGATEPREINST=@$(basename $(p)).preinst-usrmerge -Dpkg.$(basename $(notdir $(p))).USRMERGEMITIGATEPOSTINST=@$(basename $(p)).postinst-usrmerge)
+
 %:
 	dh $@
diff --minimal -Nru systemd-256.1/debian/systemd-container.postinst systemd-256.1/debian/systemd-container.postinst
--- systemd-256.1/debian/systemd-container.postinst	1970-01-01 01:00:00.000000000 +0100
+++ systemd-256.1/debian/systemd-container.postinst	2024-06-19 14:51:07.000000000 +0200
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+set -e
+
+#USRMERGEMITIGATEPOSTINST#
+
+#DEBHELPER#
+
+exit 0
diff --minimal -Nru systemd-256.1/debian/systemd-container.preinst systemd-256.1/debian/systemd-container.preinst
--- systemd-256.1/debian/systemd-container.preinst	1970-01-01 01:00:00.000000000 +0100
+++ systemd-256.1/debian/systemd-container.preinst	2024-06-19 14:51:07.000000000 +0200
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+set -e
+
+#USRMERGEMITIGATEPREINST#
+
+#DEBHELPER#
+
+exit 0
diff --minimal -Nru systemd-256.1/debian/systemd-container.usrmergemitigate systemd-256.1/debian/systemd-container.usrmergemitigate
--- systemd-256.1/debian/systemd-container.usrmergemitigate	1970-01-01 01:00:00.000000000 +0100
+++ systemd-256.1/debian/systemd-container.usrmergemitigate	2024-06-19 14:51:07.000000000 +0200
@@ -0,0 +1,5 @@
+/lib/systemd/system/systemd-sysupdate-reboot.service
+/lib/systemd/system/systemd-sysupdate-reboot.timer
+/lib/systemd/system/systemd-sysupdate.service
+/lib/systemd/system/systemd-sysupdate.timer
+/lib/systemd/systemd-sysupdate
diff --minimal -Nru systemd-256.1/debian/systemd-cryptsetup.postinst systemd-256.1/debian/systemd-cryptsetup.postinst
--- systemd-256.1/debian/systemd-cryptsetup.postinst	1970-01-01 01:00:00.000000000 +0100
+++ systemd-256.1/debian/systemd-cryptsetup.postinst	2024-06-19 14:51:07.000000000 +0200
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+set -e
+
+#USRMERGEMITIGATEPOSTINST#
+
+#DEBHELPER#
+
+exit 0
diff --minimal -Nru systemd-256.1/debian/systemd-cryptsetup.preinst systemd-256.1/debian/systemd-cryptsetup.preinst
--- systemd-256.1/debian/systemd-cryptsetup.preinst	1970-01-01 01:00:00.000000000 +0100
+++ systemd-256.1/debian/systemd-cryptsetup.preinst	2024-06-19 14:51:07.000000000 +0200
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+set -e
+
+#USRMERGEMITIGATEPREINST#
+
+#DEBHELPER#
+
+exit 0
diff --minimal -Nru systemd-256.1/debian/systemd-cryptsetup.usrmergemitigate systemd-256.1/debian/systemd-cryptsetup.usrmergemitigate
--- systemd-256.1/debian/systemd-cryptsetup.usrmergemitigate	1970-01-01 01:00:00.000000000 +0100
+++ systemd-256.1/debian/systemd-cryptsetup.usrmergemitigate	2024-06-19 14:51:07.000000000 +0200
@@ -0,0 +1,21 @@
+/lib/systemd/system-generators/systemd-cryptsetup-generator
+/lib/systemd/system-generators/systemd-integritysetup-generator
+/lib/systemd/system-generators/systemd-veritysetup-generator
+/lib/systemd/system/cryptsetup-pre.target
+/lib/systemd/system/cryptsetup.target
+/lib/systemd/system/initrd-root-device.target.wants/remote-cryptsetup.target
+/lib/systemd/system/initrd-root-device.target.wants/remote-veritysetup.target
+/lib/systemd/system/integritysetup-pre.target
+/lib/systemd/system/integritysetup.target
+/lib/systemd/system/remote-cryptsetup.target
+/lib/systemd/system/remote-veritysetup.target
+/lib/systemd/system/sysinit.target.wants/cryptsetup.target
+/lib/systemd/system/sysinit.target.wants/integritysetup.target
+/lib/systemd/system/sysinit.target.wants/veritysetup.target
+/lib/systemd/system/system-systemd\x2dcryptsetup.slice
+/lib/systemd/system/system-systemd\x2dveritysetup.slice
+/lib/systemd/system/veritysetup-pre.target
+/lib/systemd/system/veritysetup.target
+/lib/systemd/systemd-cryptsetup
+/lib/systemd/systemd-integritysetup
+/lib/systemd/systemd-veritysetup
diff --minimal -Nru systemd-256.1/debian/systemd-repart.postinst systemd-256.1/debian/systemd-repart.postinst
--- systemd-256.1/debian/systemd-repart.postinst	1970-01-01 01:00:00.000000000 +0100
+++ systemd-256.1/debian/systemd-repart.postinst	2024-06-19 14:51:07.000000000 +0200
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+set -e
+
+#USRMERGEMITIGATEPOSTINST#
+
+#DEBHELPER#
+
+exit 0
diff --minimal -Nru systemd-256.1/debian/systemd-repart.preinst systemd-256.1/debian/systemd-repart.preinst
--- systemd-256.1/debian/systemd-repart.preinst	1970-01-01 01:00:00.000000000 +0100
+++ systemd-256.1/debian/systemd-repart.preinst	2024-06-19 14:51:07.000000000 +0200
@@ -0,0 +1,9 @@
+#!/bin/sh
+
+set -e
+
+#USRMERGEMITIGATEPREINST#
+
+#DEBHELPER#
+
+exit 0
diff --minimal -Nru systemd-256.1/debian/systemd-repart.usrmergemitigate systemd-256.1/debian/systemd-repart.usrmergemitigate
--- systemd-256.1/debian/systemd-repart.usrmergemitigate	1970-01-01 01:00:00.000000000 +0100
+++ systemd-256.1/debian/systemd-repart.usrmergemitigate	2024-06-19 14:51:07.000000000 +0200
@@ -0,0 +1,4 @@
+/bin/systemd-repart
+/lib/systemd/system/initrd-root-fs.target.wants/systemd-repart.service
+/lib/systemd/system/sysinit.target.wants/systemd-repart.service
+/lib/systemd/system/systemd-repart.service