Bug#1036568: Design Defect in systemd-poweroff.service (Race Condition)

[Alex Meyer]

Package: systemd
Version: 247.3-7+deb11u2

I'm running Debian bullseye with systemd.  I also have installed mdadm 
and nut-server.

Looking in /lib/systemd/system-shutdown, I see two drop-in files with 
contents as follows:

mdadm.shutdown:

#!/bin/sh
# We need to ensure all md arrays with external metadata
# (e.g. IMSM, DDF) are clean before completing the shutdown.
/sbin/mdadm --wait-clean --scan

nutshutdown:

#!/bin/sh
/sbin/upsmon -K >/dev/null 2>&1 && /sbin/upsdrvctl shutdown

The man page for systemd-poweroff.service explains that:

All executables in this directory are executed in parallel

This is the problem.

mdadm.shutdown is trying to delay the shutdown action until data 
integrity can be guaranteed (for arrays with external metadata). 
Meanwhile, nutshutdown will immediately turn off the power.  These are 
obviously incompatible.

If execution were changed from parallel to sequential with a 
deterministic ordering, this race condition could be avoided.

There may also be other ways to address the problem, such as relocating 
the mdadm stuff to some other spot in the systemd world.  This is 
speculation on my part, as I'm no expert on systemd.

I should note that I do not personally have any md arrays with external 
metadata.  I do, however, have a UPS powering my Debian box and I can 
verify that orderly shutdown causes it to de-energize immediately at 
this point.  I think it's self-evident that waiting for anything is 
doomed to fail when run in parallel with powering off the host.

So, to be clear, this is a design defect I noticed, not a problem I'm 
experiencing.  Still, it looks like a doozy, and I feel compelled to 
report it.

I see this as a structural problem.  It's clearly documented as the 
intended behavior.  It doesn't depend on specific contents of my mdadm 
or nut configuration files, nor on my specific kernel.  Nevertheless:

# cat /etc/debian_version
11.7
# uname -a
Linux vault 5.10.178-alex #8 SMP Fri May 19 15:44:12 PDT 2023 x86_64 
GNU/Linux