Bug#884662: reproducible-builds.org: regular files sometimes treated as directories

[Simon McVittie]

Control: reassign 884662 jenkins.debian.org
Control: retitle 884662 reproducible-builds.org: regular files sometimes treated as directories
Control: severity 884662 normal
Control: user jenkins.debian.org at packages.debian.org
Control: usertags 884662 = reproducible

On Mon, 18 Dec 2017 at 23:30:05 +0000, Simon McVittie wrote:
> dpkg-deb: building package 'libglib2.0-data' in '../libglib2.0-data_2.54.2-2_all.deb'.
> tar: ./usr/share/locale/en_CA/LC_MESSAGES/glib20.mo/: Cannot savedir: Not a directory

This seems to be a symptom of some more general problem on the
reproducible-builds builders - I would guess it's either the
(FUSE?) filesystem, or a LD_PRELOAD hack that intercepts stat(), like
fakeroot does. Build logs indicate that regular files are sometimes
treated as directories by install(1), resulting in the build being
incomplete and unreproducible even in cases where it doesn't FTBFS:

https://tests.reproducible-builds.org/debian/rb-pkg/unstable/i386/glib2.0.html

--- b1/build.log	2017-12-21 23:47:26.259266661 +0000
+++ b2/build.log	2017-12-22 00:01:34.693734003 +0000
@@ -14556,6 +14571,7 @@
  /usr/bin/install -c -m 644 ./html/api-index-full.html
  /usr/bin/install -c -m 644 ./html/ch01s02.html
  /usr/bin/install -c -m 644 ./html/chapter-gobject.html
+/usr/bin/install: omitting directory './html/chapter-gobject.html'
  /usr/bin/install -c -m 644 ./html/chapter-gtype.html
  /usr/bin/install -c -m 644 ./html/chapter-intro.html
  /usr/bin/install -c -m 644 ./html/chapter-signal.html
@@ -14599,7 +14615,6 @@
  /usr/bin/install -c -m 644 ./html/left.png
  /usr/bin/install -c -m 644 ./html/pr01.html
  /usr/bin/install -c -m 644 ./html/pt01.html
-/usr/bin/install: omitting directory './html/pt01.html'
  /usr/bin/install -c -m 644 ./html/pt02.html
  /usr/bin/install -c -m 644 ./html/pt03.html
  /usr/bin/install -c -m 644 ./html/right-insensitive.png

(In a correct build, all of those files are regular files produced by
gtk-doc, and none are directories or symlinks.)

I am not aware of any reason why gtk-doc would produce
./html/chapter-gobject.html or ./html/pt01.html that are sometimes a
regular file and sometimes a directory, and I've never seen this happen
on the production buildds, so I suspect this might be some issue with
the filesystem or LD_PRELOADs used on the reproducible builders
non-deterministically producing incorrect stat() results that make
install(1) and tar(1) do the wrong thing. What filesystem is used for
the build, and which LD_PRELOAD hacks are applied?

The next upload of glib2.0 is going to have Rules-Requires-Root: no,
which will mitigate this if it's a problem with the implementation of
fakeroot used on these builders (but I'm not going to upload that until
after Christmas unless glib2.0 develops a new RC bug, because we need
to let the current version migrate so it won't block the rest of GNOME).

Regards,
    smcv