Bug#1078929: glib2.0 breaks architecture bootstrap

[Helmut Grohne]

Hi Jeremy and Simon,

On Sun, Aug 18, 2024 at 02:26:36PM +0100, Simon McVittie wrote:
> > debian/rules:30: *** Cross-compiling gobject-introspection requires host endianness = build endianness.  Stop.
> 
> Now that we're running the host g-ir-compiler under qemu, instead of
> running the build architecture g-ir-compiler with altered search paths
> (which turned out to be wrong anyway), it might be possible to remove
> this check. Perhaps you could try it, diff the resulting typelibs
> vs. natively-built ones and see whether that's the case?

>From my pov, we really should get rid of the need for qemu in the
bootstrap context. If running natively is a requirement on the GLib
side, then that's what we should accommodate on the bootstrap side if
possible. qemu is not capable of emulating all relevant architectures
and will not be in the foreseeable future.

> Failing that, am I right to think that being able to bootstrap or
> cross-compile an architecture when we already have at least one other
> architecture of the same endianness is a lesser evil than being unable
> to bootstrap or cross-compile any architecture at all?

Practically speaking, I expect this approach to be another nail in big
endian's coffin. I'd have to source a big endian machine for QA
(difficult). But then, all of the new architectures have been little
endian, so maybe big endian really is dead already? With s390x being the
only big endian release architecture left, we also have no architecture
to bootstrap it from in the event of need. I'm hoping for a better
solution and note that on a technical level this really should work
somehow as it was working before src:glib2.0 was restructured.

> There is a limit to how much I can accommodate unconditional demands:
> we're constrained by how GLib has been designed and how it works, and
> if other low-level packages have chosen to depend on it, then that's
> just something that we will have to live with.

I would agree if the problem were caused by other parties adding
dependencies on GLib, but that's not what has been happening here.
pkg-config was replaced by pkgconf removing one dependency. Due to our
conversation, I proposed dropping GLib dependencies from libxt and
cdebconf. To me, it very much is the restructuring that has "caused" the
problem (and fixed a number of other problems that I don't understand
well).

And even as I claim this would be caused by restructuring glib2.0, I am
actively looking for ways to accommodate the restructuring instead of
considering revert as an option e.g. by sending patches to libxt and
cdebconf. I am fully aware that the restructuring was not done without
reason and that the cross and bootstrap causes were considered when
doing it. The ecosystem simple has grown too big to foresee such
interactions. I try to send most of my bug reports with patches, but the
glib2.0 ones tend to be of a nature where solving them requires more
knowledge than I poses. Hence, I try to communicate the problem with as
much detail as I am able to provide for others to chime in evicting
responses like Andrea proposing to consider muon-meson.

> > Do you see any way of splitting libglib2.0-0t64 and libglib2.0-dev into
> > more granular packages with the goal of being able to cross build
> > something smaller for another endianess? In order to be useful, it would
> > have to be split in such a way that we can actually reduce a build
> > dependency of relevant packages to the smaller thing.

> One obvious choice would be to split up libglib2.0-dev and
> libglib2.0-dev-bin so that the (former) GObject-Introspection toolchain
> is separated from the rest of GLib. The qemu dependency goes with
> the formerly-GObject-Introspection parts, but I didn't want to take
> another trip through NEW (with the associated risk of having the ftp team
> reject it, or hold half of Debian hostage until we make debian/copyright
> sufficiently verbose) to add a second -dev and a second -dev-bin without
> knowing that it would actually solve anything for your use-case.
> 
> The GObject-Introspection-derived parts are, in libglib2.0-dev:
> 
> - /usr/lib/${multiarch}/glib-2.0/deb-can-run
> - /usr/lib/${multiarch}/glib-2.0/gi-*
> - /usr/bin/${gnu_type}-gi-compile-repository
> - /usr/bin/${gnu_type}-gi-decompile-typelib
> - /usr/bin/${gnu_type}-gi-inspect-typelib
> 
> and in libglib2.0-dev-bin:
> 
> - /usr/bin/gi-compile-repository
> - /usr/bin/gi-decompile-typelib
> - /usr/bin/gi-inspect-typelib
> 
> plus the corresponding man pages.

Let me skip the other choices as I agree with your recommendation.

> I suspect that the most useful split point for your purposes would be to
> break out the GObject-Introspection parts, the second-most-useful
> would be between Gio and everything lower-level, and the third-most-useful
> would be between GObject and everything lower-level.

As far as I understand things, splitting GObject-Introspection would
vaguely resemble the bookworm-state of things (with different package
names). Would you agree to that characterization?

> If we do split the -dev or shared library packages, I would suggest
> naming the split packages according to the *highest*-level component that
> they contain (in practice probably Gio or GObject), so that we would still
> have the option of breaking out even-lower-level parts afterwards without
> introducing extra-confusing package names.

Can I ask you for the favour (not demand) of going into more detail as
to how the sketched GObject-Introspection split would work out in
practice?

Evidently, we need a new binary package. As you say, we should name it
according to the highest level component, so I guess that libglib2.0-dev
would be replaced by libgobject-introspection-dev and another library
containing what libglib2.0-dev formerly contained minus
GObject-Introspection. Would that new package be called libgio2.0-dev?

Given that libglib2.0-dev has more than 3000 reverse dependencies, I
think we need to continue to provide it with that name and it needs to
provide the maximum featureset (or we cause an annoying number of
FTBFS).

You also take issue with the need to go through NEW and I think we can
get away without going through NEW by adding packages in build profiles.
Say we were to add a new build profile
pkg.glib2.0.nogobjectintrospection. Now the regular build would be kept
as it is now in unstable (minimizing the possible breakage for reverse
dependencies and avoiding to go through NEW), but we add some provides.
For instance, libglib2.0-dev would provide libgobject-introspection-dev
and libgio2.0-dev (as virtual packages). Things change when we enable
the new build profile. libglib2.0-dev would no longer be built. Instead,
libgio2.0-dev would be built as a real package and it would not provide
libglib2.0-dev nor libgobject-introspection-dev. Unless I am mistaken,
this split does not require changes to libglib2.0-0t64. Does this make
remote sense to you? If yes, would you like to receive a patch
implementing this?

If this happens to work practically, we may demote dependencies on
libglib2.0-dev to lower levels and thus enable them to to work with the
split package on a case-by-case scenario. For instance, libverto only
requires GLib core and its dependency can be demoted. Given that this
used to work in bookworm, I am hopeful that such a split would
practically work out.

Whilst this sounds convenient on the bootstrapping side, I also see that
such a split adds non-trivial costs to the glib2.0 packaging.  As such,
it incurs a permanent maintenance cost and we need to carefully consider
whether such a cost is justified by the expected benefits or whether
alternatives (e.g. trying harder at pushing GLib into the native phase)
are feasible. Splitting libverto-dev or figuring out how to build krb5
without libverto also are options to investigate.  For libxt and
cdebconf making glib2.0 optional has low extra cost, but libverto would
have to through NEW, so I am hesitant to propose a patch there before
understanding the cost on the glib2.0 side in a better way.

The most fundamental choice in this is whether glib2.0 is to be built in
the cross phase of architecture bootstrap or in the native phase.
Choosing the native phase eliminates issues about emulation, but
requires us to build build-essential without requiring GLib at any
point. Choosing the cross phase likely requires splitting libglib2.0-dev
and more granularly expressing what functionality is provided. Do you
also see this choice as fundamental and do you also see a lack of
further options?

Helmut