Is there still a point in installing libgcrypt to /lib instead of /usr/lib

[Simon McVittie]

On Sat, 15 Feb 2020 at 23:21:35 +0100, Guillem Jover wrote:
> Doing a proper /usr-merged migration is what we should have done from
> the beginning. I've been doing that with all the library packages I
> maintain that were under /lib. That includes acl, attr, libaio, libbsd
> and libmd, and I know others have been doing this too, so there's
> plenty of precedent with this.

To be clear, what Guillem means by "a proper /usr-merged migration"
here is changing individual library packages, so that the path to their
libraries that exists in the data.tar.* and in the dpkg database changes
from for example /lib/MA/libfoo.so.0 to /usr/lib/MA/libfoo.so.0. This is
not the same thing that a lot of other people mean by "the /usr merge".

I think we have consensus that consolidating all static OS files into /usr
(removing the distinction between /usr and the static parts of the root
filesystem) is the route that Debian is taking. I think we do not have
consensus on how that is to be achieved.

I would be grateful if people who advocate transitioning individual
packages, and people who consider the approach taken by usrmerge and
debootstrap to be sufficient, could refer to their preferred route in a
way that makes it clear which one they are advocating. Saying we should
do a transition "properly" is tautologous - of course we should! - but
when people disagree about what the proper way to do it is, it becomes an
ambiguous recommendation that doesn't guide anyone to do the right thing.

The approach that transitions individual packages solves some bugs
(notably, if you ask dpkg which package owns /usr/lib/MA/libfoo.so.0,
you get the right answer, which has a lot of desirable consequences). It
also appears to *cause* some bugs. The ones I know about are:

* packages that hard-code paths into /lib stop working on systems that
  have *not* undergone the usrmerge-style /usr merge, because
  /lib/MA/libfoo.so.0 no longer exists (for example see #950715
  involving libgcc.so.1 and the gold linker);
* there is a class of bugs on systems that have *not* undergone the
  usrmerge-style /usr merge, involving old libraries lingering in /lib/MA
  (see #949395 for a summary of the instances that I know about), which
  are very hard to debug because they are unreproducible, to do with the
  state of an individual system, and are related to upgrades that happened
  years in the past and whose logs expired long ago;
* when paths migrate between package names and between paths at the same
  time, there can be undetected file conflicts on systems that *have*
  undergone the usrmerge-style /usr merge (for example see #950624,
  again in libgcc.so.1)

The term "/usr merge" or "merged /usr" more commonly refers to making
/usr/lib and /lib, /usr/bin and /bin, etc., *equivalent* (with symlinks
/lib -> usr/lib, etc.), as implemented by the usrmerge package and
debootstrap --merged-usr.

In many other distributions, for example Fedora, there was a flag day at
which merged /usr became mandatory. This makes various classes of bugs
like #949395 and #950715 cease to exist: for example, it doesn't matter
if /lib/MA/libfoo.so.0 is hard-coded somewhere, because that path still
exists (realpath would resolve it to /usr/lib/MA/libfoo.so.0).
(However, this can't solve everything: #950624 would still have been a
bug if we had had a merged /usr flag-day.)

In Debian we have (as usual) made life more difficult for ourselves by the
/usr merge being optional, which means that in any transition or upgrade
scenario, maintainers have to consider two cases: one where the system
has undergone the /usr merge, and one where it has not. For example,
#950624 only happens on systems that have undergone the /usr merge,
while #950715 and #949395 only happen on systems that have not. Testing
on any single system cannot detect all of these: to detect all the bugs,
we have to try both configurations.

    smcv