[Pkg-xen-devel] Bug#1028251: New Patch (Was: Re: Bug#1028251: xen: FTBFS when building xen binary packages for sid on x86_64)

[Hans van Kranenburg]

Hi,

On 1/13/23 22:45, Chuck Zmudzinski wrote:
> On 1/13/23 7:39 AM, Marek Marczykowski-Górecki wrote:
>> On Fri, Jan 13, 2023 at 12:58:29AM -0500, Chuck Zmudzinski wrote:
>>> On 1/11/2023 10:58 PM, Chuck Zmudzinski wrote:
>>>> On 1/9/23 12:55 PM, Hans van Kranenburg wrote:
>>>>> Hi!
[...]
Yolo style cutting out lines here...
[...]
>>>
>>> Regarding the systemd files causing ftbfs, this explains it:
>>>
>>> https://salsa.debian.org/xen-team/debian-xen/-/blob/master/m4/systemd.m4#L119
>>>
>>> and this:
>>>
>>> https://salsa.debian.org/xen-team/debian-xen/-/blob/master/tools/configure.ac#L480
>>>
>>> The comments indicate that using AX_AVAILABLE_SYSTEMD() will
>>> by default enable systemd if systemd development files are on the
>>> build system, and AX_ALLOW_SYSTEMD() means --enable-systemd
>>> must explicitly be passed to tools/configure to enable it. Upstream
>>> uses the former, so build systems with systemd development files
>>> by default will ftbfs because that produces missing files that dh_missing
>>> in debian/rules does not like.
>>>
>>> So the reason there is ftbfs on my system is that my system has
>>> the systemd development package installed.
>>
>> By the way, maybe a better fix would be to pass --enable-systemd, add libsystemd-dev
>> build-dep and list them in the package? They might require patching to
>> support Debian-specific upgrade machinery, though...
>>
>> Not installing xendriverdomain.service is one of things missing for
>> driver domains support
>> (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=922033).
>>
> 
> Hi Marek,
> 
> I wouldn't be against fixing it that way. In fact, I would prefer
> that Debian packaged Xen with full support for native systemd units.
> I am willing to wait until if/when the package maintainers have
> full systemd support in the Xen packages.
> 
> Perhaps this is an opportunity for you to try to fix 922033 again.
> I see it has been sitting there for a few years now. Let's see
> what Hans thinks.

Yeah, well, so, the thing here is...

When Debian started to package Xen (thanks! Bastian, in 200X), the
upstream init scripts were copy pasted, and adjusted to have the ability
to have different Hypervisor-ABI-incompatible versions installed at the
same time. Also, this is related to the collection of Makefile patches
we carry around to have ABI-incompatible stuff end up in a directory
like /usr/lib/xen-4.14/ and /usr/lib/xen-4.17/ !

What does this mean? Well, in the most basic sense it means that you
could apt-get (dist-)upgrade and then still be able to xl shutdown a
domU afterwards before doing reboot, because it will choose the right
tools which match with the ABI of the *now* running hypervisor instead
of being left with a dumpster fire, which in the end causes you to shout
curse words and cause you to have to go to the machine and hold the
power button for 5 seconds to force power it off.

This is the thing about where you upgrade from Xen 4.14 to Xen 4.17
during the upgrade from Debian 11/Bullseye to Debian 12/Bookworm, it
will allow you, if booting the whole new thing is a huge failure, to
reset the computer, and in grub, choose to use the previous Xen (and
possibly do that in combination with previous Debian linux kernel) and
then have a system where you again at least can start your domUs again
*) and first have a good rest, night of sleep before starting to dig
into what's going wrong.

So, this is exactly the same way of doing stuff like how you can also
reboot back into the previous Linux kernel (ABI-compatible) one during a
system upgrade, even if you're not using Xen at all!

I like this very much. This is the kind of thing that helps admins of
systems that have just local disks and a few domUs. Like, the case where
you support some non-profit organization with their server stuff running
on donated hardware. (Yes, I also do some of those, I do!) And, in case
something does fail (there could always be something like a misbehaving
mpt3sas card in the hardware or anything that no one else spotted yet),
the admin does not have to end up in total panic mode after doing the
upgrade on a Friday afternoon lying upside down inside a broom closet,
but they can just at least recover from the situation and have something
that's running again, and then a day later, or 2 or 3 days or a week
later return on another planned moment to fix it, after asking around.

Upstream Xen stuff doesn't have anything like that.

But, they actually look at us, and they think, ooh, this is actually
nice, we should have that also by default.

The fact that we have this changed/altered/divergent init scripts in
Debian is the main reason that we cannot just enable systemd things
which will put upstream whatever on the system.

So, what could we do about this?

The project plan (that could be drafted on an A4 paper) could look like,
gather around all distro maintainers of Linux distro's that are shipping
Xen, and then search for a 'Project owner', which we totally need to be
someone that is actually employed at a company that actually cares about
getting the results of this.

Then we go look at the Debian stuff and fix upstream to do the same
thing, also fixing all the init/systemd stuff that got lost along the
way, and then we can push it down to all other distro's as well again.

And after all of that is done, there will be a time where we actually
(at Debian) can have a different response to everything init script
related than "sorry, probably won't happen".

If you look at the init script stuff in Xen upstream already, it quickly
shows that it's just a total dumpster fire. Someone cobbled up something
in the year 2005, and after that, only changes have been made by people
who actually tried to touch it for a few seconds with a 10-foot pole.

So, nobody is really caring about this. There is not an actual person
upstream who is involved into this. As long as that's the case, it's not
a healthy thing for ourselves to start to try fixing all of that from a
downstream POV.

We're currently doing 'good' with the Debian Xen Team, I think. We can
keep up with security updates, and we're in some sort of OK position to
get the essential stuff into place for Bookworm, but to say it in good
Dutch "Nee, het houdt niet over...".

Knorrie

P.S. and if you think "I have no idea what you're rambling about
Knorrie, I have never experienced that problem", well, then thank you
for using Debian ;]

*) Yeah, this works for PV and PVH, but until the #$^$%& qemu stops
using internal unstable function calls any more so that it doesn't have
to hard depend on libxenmisc4.1X any more, you're still screwed for HVM.
BUT! if you just shut down the domUs nicely and reboot and you have to
go back, then dpkg -i or whatever the previous qemu and you can still
start all domUs again instead of going into full panic mode during the
night.