moving graphical installer to GTK 3

[Simon McVittie]

On Mon, 17 May 2021 at 17:48:45 +0200, Cyril Brulebois wrote:
> I've checked what would
> happen with GTK 3 in cdebconf and cdebconf-gtk-terminal (I had forgotten
> about cdebconf-gtk-entropy until writing this reply).

I think it's much too late in the Debian 11 cycle to be doing this for
Debian 11, unless the relayout loop turns out to be completely impossible
to fix any other way.

My biggest concern about cdebconf and GTK 3 is that it's relying on the
ability to call into GTK APIs from more than one thread. GTK 2 tried to
support this pattern, with gdk_threads_enter() and gdk_threads_leave()
providing locking, but it turned out not to be a good idea. GTK 3
deprecates and strongly discourages these, and says:

    All GDK and GTK+ calls should be made from the main thread

I think the good way to do it would be to have a debconf thread that
speaks the debconf protocol (which requires blocking in some places),
have a GTK thread that runs the GTK/X11 event loop (which requires not
blocking for extended periods), and pass messages between them with GLib's
GMainContext mechanism, which lets you schedule a function to be called
"a bit later" in the other thread. The GTK thread would run the default
GMainContext (as in g_main_context_default()), and behave like the main
thread from GTK's point of view. The debconf thread would *actually*
be the main thread, but have its own separate GMainContext, which it would
iterate when it's waiting for the GTK thread to do something. I'm familiar
with this model from writing non-GUI event-driven code using GLib.

Philip Withnall wrote a really good introduction to GMainContext which
is now part of the GLib API reference:
https://developer.gnome.org/programming-guidelines/unstable/main-contexts.html.en

The GTK 3 frontend for reportbug uses this model - reportbug is in the
same position as cdebconf, with an externally-imposed programming model
based on calling functions that must block until they have an answer,
but simultaneously needing to run a GUI that must not block.

I spent a bit of time trying to annotate which cdebconf-gtk functions
are called from which thread, and I don't think it would be impossible to
disentangle. Ideally, for each function and 95% of the variables/struct
members, it should be obvious whether the function is called/the variable
is accessed from the debconf thread (only!) or from the GTK thread
(only!). This avoids needing explicit locking/mutexes/condvars/atomics
most of the time; the thread-safe operations are implicit in the
message-passing between one GMainContext and the other.

Obviously this would require a lot of refactoring, which should be done
at the beginning, not the end, of a release cycle - but I don't think a
high-quality port to GTK 3 is going to be feasible as something that is
worked on during hard freeze anyway.

I also think the beginning of Debian 12 would be a good time to reconsider
whether the graphical d-i mode is really the best way for non-expert
users to install Debian. The restricted capabilities of udebs make d-i
quite a "one hand tied behind your back" environment, which was still
a necessary evil a few years ago; but now that systems with 512M RAM
are literally given away with a magazine, perhaps that's becoming less
necessary than it once was?

For embedded and server uses, which have the tightest RAM restrictions,
or for many of the use-cases of preseeding, perhaps imaging the system
from a preprepared disk image is a better route? We have official cloud
images, and lots of ways to make your own, all of which start from
debootstrap or equivalent. The traditional TUI flavour of d-i also
seems good for this use: it's perhaps less crucial to have full support
for non-Latin character sets in an environment where your interactions
with the installed system will have more in common with 1980s Unix than
any particular natural language.

For desktop- and laptop-class systems, I think Calamares from
a live-system environment has a lot of potential, perhaps combined with
gnome-initial-setup or a non-GNOME equivalent to do "first-run" setup
on the installed system. This is probably the place where a graphical
installer with many languages and a familiar UX is most important.

We're probably not ready for what EndlessOS does - their installer is
basically dd with a progress bar, and *all* customization is done after
rebooting into the installed system - but it's worth considering how
close we could get. The big advantages of doing that (and the reasons
why this is how phones and OEM installations of Windows/macOS work) are
that it's trivial to replicate the un-customized installation across
any number of machines either for yourself or to be handed to others,
and that all the GUI widgets, fonts, input behaviours, design language
etc. are automatically the same as in the installed system, because it
literally *is* the installed system.

> The installer seems to be working somewhat. I'm seeing strange things
> regarding layout, regarding widget expansion (basically we have some
> wasted vertical space).

The layout algorithm was basically rewritten at some point, so I think
we can expect it to have somewhat different rendering. (This is sort
of the point; the only reason you even considered this for Debian 11 is
that the undocumented layout code in GTK 2 is going into a loop.)

GTK 2 theming was done with an ad-hoc style language or programmatically
by an "engine", but GTK 3 themes are basically just CSS, so it might well
be possible to fudge the layout a bit by adjusting margins/padding if
there are particular problems.

I see d-i uses Clearlooks (the former GNOME default engine) for normal
mode or hc as a high-contrast mode. For GTK 3, I expect we'd want to use
the GTK/GNOME default theme Adwaita, or the light or dark high-contrast
accessibility themes; all of those are part of GTK 3 itself and do not
need extra modules.

Talking of themes, the parts of d-i that write out configuration
files like .gtkrc will probably need changes. GTK 3 mostly uses GLib's
GSettings abstraction for settings storage. On an installed system,
this reads and writes the dconf database, using the dconf daemon to
mediate writes, but for the single-GUI-process environment of d-i,
setting the GSETTINGS_BACKEND environment variable to request that
settings are stored in a GKeyFile (simple .ini-style text) seems more
appropriate. The keyfile backend is part of GIO, so it should be in
libglib2.0-0-udeb already.

> I'm also seeing a different behaviour regarding
> the expose (GTK 2) vs. draw (GTK 3) event handling, meaning the banner
> doesn't get repainted automatically

I would hope that the banner can be drawn in a more GTK-3-friendly way,
which might well end up being less code too.

A GtkLabel subclass would probably work - if you were trying to create
this UX on the web, you'd make it <div id="banner">Rescue mode</div>,
and use CSS to make its size and background image what you want.
I think GTK 3's CSS-based layout and styling can do more or less the same.

>     May 17 15:28:46 debconf: cdebconf_gtk (process:257): GLib - DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created

This might be more serious than you think, because cdebconf has a debconf
thread that speaks the debconf protocol, and a UI thread that runs
the GTK event loop, and there's nothing to stop them from both calling
getenv() concurrently, so editing the environment (a global variable)
is not necessarily going to end well.

If the environment variable is being set in order to pass it to a
subprocess, then using things like g_listenv() and g_environ_setenv() to
edit a copy of the environment, then passing that copy to the subprocess
instead, would be better.

>     May 17 15:28:46 debconf: cdebconf_gtk (process:257): VTE - WARNING: (../src/vtepty.cc:670):bool _vte_pty_spawn_sync(VtePty*, const char*, const char* const*, const char* const*, GSpawnFlags, GSpawnChildSetupFunc, gpointer, GDestroyNotify, GPid*, int, GCancellable*, GError**): runtime check failed: ((spawn_flags & forbidden_spawn_flags()) == 0)

This means cdebconf-terminal is passing GSpawnFlags into a function
that were allowed in old vte but not in new vte. I would guess that it's
calling into a deprecated/discouraged/old API and should be doing things
a bit differently now - I'm fuzzy on the details, but I think there are
parts of the VtePty API that turned out to be a bad idea and have been
deprecated, with an API at a slightly different level of abstraction as
the recommended replacement. I think it's to do with whose responsibility
it is to fork() and exec() the child.

If in doubt, do what gnome-terminal does, or do what the example src/app/
in vte2.91 does (it's basically a heavily simplified gnome-terminal).

    smcv