[Piuparts-devel] How important are empty directories?

[Helmut Grohne]

Hi Paul,

thanks for taking the time to respond.

On Tue, Aug 29, 2023 at 08:09:24PM +0200, Paul Gevers wrote:
> > In the majority of cases, such empty directories are more of a bug than
> > a feature and we can simply delete them. In some cases, they really are
> > used though.
> 
> Can you elaborate? My imagination might be limited, but all I could come up
> with is that maintainer scripts expect a directory to be there and try to
> write to it without checking (is that also the case for the issue that you
> referred to in [1] mentioned by Jochen?). I *think* that if a package needs
> the directory to installs a file into, the directory will be created (or
> will dpkg fail as it assumes the directory already exists)?

I'm not sure I understand openrc yet. It installs /lib/rc/tmp and I
hunted this for like half an hour. I guess that it uses this as a
temporary mount point for performing a pivot_root. If that is correct,
its absence makes the system unbootable.

For systemd we actually found that some cryptsetup-related autopkgtest
was failing as systemd dropped one of the empty directories. That test
has since been adapted.

So yeah, this can fail. Subtly.

> > When they are used, we need to do something about that
> > deletion and I've submitted e.g.
> > https://salsa.debian.org/systemd-team/systemd/-/merge_requests/208 where
> > I got negative feedback on the need to address this.
> 
> Which has been fixed nevertheless.

By deleting all empty directories.

> I agree that *only* pleasing piuparts is not the best time spent by
> maintainers, *if* that's really the only problem we're solving. I guess
> we're having a hard time with this check to find a real life problem. I
> *think* that https://bugs.debian.org/1050606 (linked from that MR) points at
> an example of this problem, right? So that's at least one.

The problem with not pleasing piuparts is that this can break testing
migration of *other* packages. So I think we have to do something.
Patching piuparts is a possible way forward, but that is not without
cost either. Possibly, pleasing piuparts poses a lower cost.

> If everybody agrees that the only place where this causes real life issues
> is piuparts, than I rather have piuparts ignore this problem. After we're
> fully canonized, are we safe again and could we turn the check on again?
> Looking at that example bug above though, I'm not sure we can only see this
> class of problems in piuparts.

Yes, once we're done we can turn the check back on.

> > On the flip side, systemd has been the last package for me to file a
> > patch where this issue has practical consequences already. All others
> > have been filed already. Beyond these, there are five more cases on the
> > horizon that likely need to be mitigated when we lift the moratorium.
> 
> With patches ready?

Those five ones don't have patches yet. I started looking and at least
firmware-b43-installer is a non-issue, because its postinst has

    mkdir -p "${FIRMWARE_INSTALL_DIR}/${B43}"

which is the empty directory in question, so it already is mitigated. I
guess firmware-b43legacy-installer is similar. printer-driver-foo2zjs is
likely affected for real, but a simple mkdir in its maintainer script
(without the trigger mess) can save it as no other package contains
/lib/firmware/hp. Other than openrc, there is netplan-generator which
likely isn't worse than pkgconf-bin. In any case, I think you may assume
that patches appear in time.

> > So while the mitigation is ugly, the low number of affected packages and
> > the temporary nature of the mitigation made me conclude that doing this
> > is a reasonable trade-off. Do you agree?
> > 
> > Another example for the ugliness is #1050412.
> 
> Will the next time that pkgconf-bin is installed (reinstall or upgrade)
> recreate the directory again (without your patch)? Or will the directory be
> lost forever?

When the cause for loosing a directory has been resolved (i.e. no
package installs files into the corresponding aliased location), any
reinstall or upgrade of the affected package will restore empty
directories. So the problem is self-healing over time.

> I'm not totally sure you got the answer to the question you raised, but I'm
> also not totally sure what you wanted to hear.

Not the answers I wanted, but still moving forward. I think we either
need a volunteer for patching piuparts or bump these issues to RC later.
That is bumping pcp, libswe-dev and pkgconf-bin and figuring those other
five (see above). I think that creating the patches is less work than
producing a piuparts patch, but that only matters if you don't want to
hack piuparts yourself.

Helmut