[Pkg-zsh-devel] Bug#980903: debhelper: doc-base doc-id deduplication does not work as documented with multiple "dh_installdocs -p<pkg>" calls; causes /usr/share/doc-base/<doc-id> to be installed multiple times

[Axel Beckert]

Package: debhelper
Version: 13.3.1
Severity: serious
Justification: Causes file conflicts within binary packages of the same source package
Control: block 961757 by -1

While trying to solve https://bugs.debian.org/961757 ("Please install
the FAQ") in zsh, I ran into the following IMHO severe issue reported by
lintian at "warning level":

W: zsh source: binaries-have-file-conflict zsh-common zsh-doc usr/share/doc-base/zsh-faq

Scenario: I have two documents which I want to register with doc-base:
the Zsh Manual and the Zsh FAQ. Both are available in different file
formats. The FAQ is split over two binary packages (plain text in
zsh-common, HTML in zsh-doc) and I tried to use this technique described
in dh_installdocs man page:

       debian/package.doc-base.*
           If your package needs to register more than one document, you
           need multiple doc-base files, and can name them like this. In
           the event that multiple doc-base files of this style in a
           single source package share the same doc-id, they will be
           installed to usr/share/doc-base/package-* instead of
           usr/share/doc-base/doc-id.

But this deduplication does not happen with zsh as the lintian warning
cited above shows.

I tried different file name combinations. Initially I had:

$ find debian -name '*doc-base*' | xargs fgrep Document:
debian/zsh-common.doc-base.faq:Document: zsh-faq
debian/zsh-doc.doc-base.faq:Document: zsh-faq
debian/zsh-doc.doc-base.manual:Document: zsh

Since there's only one document to register with in zsh-common, I tried
to use zsh-common.doc-base instead of zsh-common.doc-base.faq:

$ find debian -name '*doc-base*' | xargs fgrep Document:
debian/zsh-common.doc-base:Document: zsh-faq
debian/zsh-doc.doc-base.faq:Document: zsh-faq
debian/zsh-doc.doc-base.manual:Document: zsh

In both cases, there was indeed one file twice in the built binary
packages, as expected with different size:

$ debc | fgrep zsh-faq
-rw-r--r-- root/root       504 2021-01-24 00:45 ./usr/share/doc-base/zsh-faq
-rw-r--r-- root/root       519 2021-01-24 00:45 ./usr/share/doc-base/zsh-faq

After running the build with DH_VERBOSE=1, I found the reason: zsh's
debian/rules calls dh_installdocs multiple times with the -p<package>
option due to different options:

override_dh_installdocs-indep:
	dh_installdocs -pzsh-doc --link-doc=zsh-common --doc-main-package=zsh-common
	dh_installdocs -pzsh-common

But the documented deduplication only happens within a single
dh_installdocs call and is not spread over multiple calls.

So IMHO the value of %used_doc_ids needs to be cached somewhere under
debian/.debhelper/ or similar and if that cache file exists, it needs to
be read upon every dh_installdocs invocation.

Setting severity to RC as I think that debhelper should not be released
in bullseye with this issue being present, except if (worst case)
documented in dh_installdocs(1) as exception or (obviously) if fixed. (I
think being one of the top 5 committers of debhelper, I still can claim
that this is the maintainer's opinion, even though I haven't really
committed something for a few years and still have way less commits than
Niels has. I also intend to have a look into a solution, but if Niels is
quicker, he's of course free to fix it first. Hints are also
appreciated. ;-)

P.S.: The zsh package triggering this issue can be found in git in the
install-faq-961757 branch since I didn't want to commit this to the
normal packaging branch (called "debian" in this repo):
https://salsa.debian.org/debian/zsh/-/tree/install-faq-961757

P.P.S.: According to "git blame" this bug is in there since that feature
(see #525821) was added in debhelper 9.20130504. I quite surprised
that nobody else seems to have ran into it so far.

-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (990, 'unstable'), (600, 'testing'), (500, 'unstable-debug'), (500, 'buildd-unstable'), (110, 'experimental'), (1, 'experimental-debug'), (1, 'buildd-experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.10.0-1-amd64 (SMP w/4 CPU threads)
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)
LSM: AppArmor: enabled

Versions of packages debhelper depends on:
ii  autotools-dev            20180224.1+nmu1
ii  dh-autoreconf            19
ii  dh-strip-nondeterminism  1.10.0-1
ii  dpkg                     1.20.7.1
ii  dpkg-dev                 1.20.7.1
ii  dwz                      0.13+20210118-1
ii  file                     1:5.39-3
ii  libdebhelper-perl        13.3.1
ii  libdpkg-perl             1.20.7.1
ii  man-db                   2.9.3-2
ii  perl                     5.32.0-6
ii  po-debconf               1.0.21+nmu1

debhelper recommends no packages.

Versions of packages debhelper suggests:
ii  dh-make  2.202003

-- no debconf information