[Pkg-xen-devel] Bug#1028251: New Patch (Was: Re: Bug#1028251: xen: FTBFS when building xen binary packages for sid on x86_64)

Fri Jan 13 05:58:29 GMT 2023

On 1/11/2023 10:58 PM, Chuck Zmudzinski wrote:
> On 1/9/23 12:55 PM, Hans van Kranenburg wrote:
> > Hi!
> > 
> > On 09/01/2023 18:44, Chuck Zmudzinski wrote:
> >> Control: tag -1 + moreinfo
> >> 
> >> thanks
> >> 
> >> On 1/9/23 8:09 AM, Hans van Kranenburg wrote:
> >>> Hi Chuck,
> >>>
> >>> On 1/8/23 23:18, Chuck Zmudzinski wrote:
> >>>> [...]
> >>>>
> >>>> The build failed:
> >>>>
> >>>>    debian/rules override_dh_missing
> >>>> make[1]: Entering directory '/home/chuckz/sources-sid/xen/xen-4.17.0'
> >>>> dh_missing --list-missing
> >>>> dh_missing: warning: usr/lib/modules-load.d/xen.conf exists in debian/tmp but is not installed to anywhere
> >>>> dh_missing: warning: usr/lib/systemd/system/proc-xen.mount exists in debian/tmp but is not installed to anywhere
> >>>> dh_missing: warning: usr/lib/systemd/system/xen-init-dom0.service exists in debian/tmp but is not installed to anywhere
> >>>> dh_missing: warning: usr/lib/systemd/system/xen-qemu-dom0-disk-backend.service exists in debian/tmp but is not installed to anywhere
> >>>> dh_missing: warning: usr/lib/systemd/system/xen-watchdog.service exists in debian/tmp but is not installed to anywhere
> >>>> dh_missing: warning: usr/lib/systemd/system/xenconsoled.service exists in debian/tmp but is not installed to anywhere
> >>>> dh_missing: warning: usr/lib/systemd/system/xendomains.service exists in debian/tmp but is not installed to anywhere
> >>>> dh_missing: warning: usr/lib/systemd/system/xendriverdomain.service exists in debian/tmp but is not installed to anywhere
> >>>> dh_missing: warning: usr/lib/systemd/system/xenstored.service exists in debian/tmp but is not installed to anywhere
> >>>
> >>> I cannot reproduce this error here locally and the CI build also succeeds:
> >>>
> >>> https://salsa.debian.org/xen-team/debian-xen/-/pipelines/481577
> >> 
> >> I thought I had a fairly clean sid install, but I think the problem
> >> on my system could be caused by some obscure grandfathered in
> >> setting because the sid I am using was updated from all the way back to
> >> an original install of jessie many years ago...
> >> 
> >> It might be time for me to refresh my sid with a clean installation.
> >> 
> >> Out of curiosity and if you have time, can you answer a couple of
> >> question if you know the answer?
> >> 
> >> 1. Do the builds on a clean environment produce the missing files
> >> listed in my build?
> > 
> > No, after my local package build, there's no such things in there:
> > 
> > ~/build/xen/debian-xen/debian/tmp/usr/lib m (master) 1-$ ll
> > total 0
> > drwxr-xr-x 1 knorrie knorrie  110 Jan  8 23:51 debug
> > drwxr-xr-x 1 knorrie knorrie 2048 Jan  8 23:50 x86_64-linux-gnu
> > drwxr-xr-x 1 knorrie knorrie   20 Jan  8 23:51 xen-4.17
> > 
> >> 
> >> 2. Are those systemd service files installed anywhere in the xen
> >> binary packages, either in arch=x86_64 packages or for the arch=all
> >> packages such as xen-utils-common?
> > 
> > No, they are not:
> > 
> > https://packages.debian.org/search?searchon=contents&keywords=xenconsoled.service&mode=path&suite=unstable&arch=any
> > 
> >> If you don't know the answer to these questions I will investigate
> >> myself to find the answers, so you can work on more important things.
> >> 
> >>>
> >>> How are you building the packages? In a clean build environment, using
> >>> for example sbuild or pbuilder, or in an environment where unrelated
> >>> other build dependencies could be present, that are not included in the
> >>> xen list, but maybe 'wake up and do something' if they're present?
> >> 
> >> As I said, I am building on a sid install that might have some
> >> stuff grandfathered in from old releases going back to jessie.
> >> I also might have some stale stuff around from my private builds
> >> of the traditional device model available from xen that is not
> >> part of the Debian packages. I will investigate these possible causes.
> >> 
> >> I use debuild as a frontend to dpkg-buildpackage to build the packages.
> > 
> > Yes. So (I'm not entirely sure how it works, but as example, just making
> > something up here): After doing something else first, you might end up
> > with a system that has for example dh-systemd-yolo-all-the-things-helper
> > installed. And, it might be that only it being present means that the
> > package build process changes. It might even be a 'feature' of that
> > helper... "just add it to your build depends, and it will automatically
> > do all the things for you!!!~``1"
> > 
> > This is why it is very much recommended to build the packages using
> > something like sbuild, so that you can be sure that every time it will
> > start with a super minimal chroot which only has some essential things,
> > and that the only build dependencies used will be the ones that are
> > explicitly defined in the debian/control of the package.
> >>> You can also compare your own build output with the full one from the CI
> >>> job:
> >>>
> >>> https://salsa.debian.org/xen-team/debian-xen/-/jobs/3767564/raw

Regarding the systemd files causing ftbfs, this explains it:

https://salsa.debian.org/xen-team/debian-xen/-/blob/master/m4/systemd.m4#L119

and this:

https://salsa.debian.org/xen-team/debian-xen/-/blob/master/tools/configure.ac#L480

The comments indicate that using AX_AVAILABLE_SYSTEMD() will
by default enable systemd if systemd development files are on the
build system, and AX_ALLOW_SYSTEMD() means --enable-systemd
must explicitly be passed to tools/configure to enable it. Upstream
uses the former, so build systems with systemd development files
by default will ftbfs because that produces missing files that dh_missing
in debian/rules does not like.

So the reason there is ftbfs on my system is that my system has
the systemd development package installed.

After doing:

chuckz at debian:~$ sudo apt-get remove libsystemd-dev

the build succeeded on my sid system even with

dh_missing --fail-missing

in debian/rules.

Since systemd files are not installed in the packages, there
is no need to build them. So you might consider this patch
to explicitly disable systemd with the override_dh_auto_configure
setting until the packages include the native systemd startup files
that fixes this ftbfs even if libsystemd-dev is installed:

--- a/debian/rules    2022-12-21 16:34:51.000000000 -0500
+++ b/debian/rules    2023-01-12 20:49:35.282125205 -0500
@@ -206,6 +206,7 @@
         --with-libexec-leaf-dir=xen-$(upstream_version) \
         --disable-blktap1 \
         --disable-blktap2 \
+        --disable-systemd \
         --disable-qemu-traditional --disable-rombios \
         --with-system-qemu=/usr/libexec/xen-qemu-system-i386 \
         --enable-ovmf --with-system-ovmf=/usr/share/ovmf/OVMF.fd \

So it turns out the libnl-route-3-dev package is *not* the culprit of
ftbfs, so I was wrong about that in my earlier message. I would consider
that if you apply the above patch, you could mark this bug as done.

I don't know if you consider the existence of the libnl-route-3-dev
package on the build system causing remus/COLO support being
added a minor reproducibility bug:

On my build:

checking for LIBNL3... yes
checking for SYSTEMD... no

On the CI build:

checking for LIBNL3... no
configure: WARNING: Disabling support for Remus network buffering and COLO.
    Please install libnl3 libraries (including libnl3-route), command line tools and devel
    headers - version 3.2.8 or higher
checking for SYSTEMD... no

Since those libnl3 libraries are not Build-Depends, the presence of
those packages on the Build system makes the build not
reproducible when compared to builds in the CI environment.
Not a big problem, though, so I think you can ignore it if you
want as long as no one complains the remus/COLO feature is
missing in the Debian Xen packages.

Thanks,

Chuck

