Bug#954783: Non-functioning gnome-apps after mate-session due to surviving CLUTTER_BACKEND env var

[Simon McVittie]

Control: reassign -1 libclutter-1.0-0,gnome-session-common,debian-mate-default-settings
Control: found -1 clutter-gtk/1.8.4-4
Control: found -1 gnome-session/3.30.1-2
Control: found -1 mate-session-manager/1.24.0-2

On Mon, 23 Mar 2020 at 13:32:58 +0100, Dimitri Schwarz wrote:
> In short: When running a mate session the environment-variable
> CLUTTER_BACKEND=x11 is set. This variable survives a logout and relogin into
> the GNOME wayland-session, which doesn't override it. The result is a bunch
> of not launchable clutter-gtk applications.

I think this is at least partially a debian-mate-default-settings bug. For
now I've assigned it to all three involved packages (clutter uses the
variable, MATE sets it, GNOME doesn't unset it); it might make sense to
change more than one of these packages.

Full text quoted for the MATE maintainers:

> To reproduce the issue:
>  * Install debian 10.3.0 with gnome and mate environment
>  * Optional:
>      * Login into GNOME (wayland) Session
>          * Run `env | grep -Ei 'x11|wayland'`
>             XDG_SESSION_TYPE=wayland
>             WAYLAND_DISPLAY=wayland-0
>          * Note: CLUTTER_BACKEND is not set
>          * Launch `gnome-control-center` via graphical environment or console
>             * Works fine
>          * Logout
>  * Login into MATE session
>      * `env | grep -Ei 'x11|wayland'`
>         CLUTTER_BACKEND=x11
>         XDG_SESSION_TYPE=x11
>      * `sudo grep -Ri CLUTTER_BACKEND /etc/`
>         /etc/X11/Xsession.d/99mate-environment:    export CLUTTER_BACKEND=x11
>      * Logout
>  * Login into GNOME (wayland) session
>      * `env | grep -Ei 'x11|wayland'`
>         CLUTTER_BACKEND=x11 # <- survived!
>         XDG_SESSION_TYPE=wayland
>         WAYLAND_DISPLAY=wayland-0
>      * Launch `gnome-control-center` via graphical environment or console
>      * Observe it doesn't startup
>         (gnome-control-center:4399): Clutter-Gtk-ERROR **: 12:23:42.914: *** Unsupported backend.
>      * In fact all of the following apps terminate the same way
>          * `compgen -c | while read cmd; do [ -n "$(which $cmd)" ] && ldd $(which $cmd) | grep -q clutter-gtk && echo $cmd; done`
>             cheese
>             totem
>             evolution
>             gnome-control-center
>             gnome-contacts
>             cheese
>             totem
>             evolution
>             gnome-control-center
>             gnome-contacts
>             gnome-nibbles
>             swell-foop
>             lightsoff
>             quadrapassel
>      * Now launch `CLUTTER_BACKEND=wayland gnome-control-center`
>         * Works fine
> 
> So I suppose the surviving of session-variables is the core issue here. In
> case of it beeng rather a feature than a bug, I suppose gnome should set
> CLUTTER_BACKEND explicitly aswell. Or maybe clutter-gtk shall respect this
> constellation of CLUTTER_BACKEND and GDK_WAYLAND_DISPLAY?

For context, GNOME normally provides both a Wayland compositor
(gnome-shell's native protocol) and an X11 display (Xwayland, backed by
gnome-shell). I think MATE only has an X11 display (Xorg).

Clutter is a scene-graph library that is essentially unmaintained
upstream, but unfortunately some important apps that started using it
when it was the next big thing are still using it, both within GNOME
(Totem is a good example) and outside GNOME. It is meant to be replaced
by the GSK layer in GTK 4, but GTK 4 is not yet stable, and until it
is, we don't have any properly maintained scene-graph available. Some
upstream and downstream GNOME developers keep Clutter on life-support so
that apps like Totem will still work, but our ability to make significant
changes is very limited. Clutter's supporting library Cogl is in a
similar situation.

To give you an idea of the size of the problem, Clutter is sufficiently
unmaintained that the mutter compositor used in GNOME (and Budgie)
now contains its own fork of Clutter and Cogl, with the parts that it
doesn't use deleted to reduce maintenance effort.

Meanwhile, GTK 3 prefers to use Wayland when available, via the "wayland"
backend in GDK. It can also use X11, via the "x11" backend (which will
be used in MATE and other X11-only desktop environments, or in GNOME
when Wayland is disabled, or when forced with GDK_BACKEND=x11).

Clutter *also* works on either Wayland or X11. It has a "new" backend,
gdk (which uses the same GDK library as GTK does), and an "old" backend,
x11 (which is X11-only). By default it will switch between backends
automatically, and I think it prefers the "gdk" backend, but it will
use x11 if forced via CLUTTER_BACKEND=x11 (as MATE currently does).

If we ever get into a situation where GTK has chosen to use Wayland,
but Clutter has chosen to use the x11 backend, then we have a problem,
because those are not a combination that can work: Clutter isn't going
to get very far by trying to integrate with a GTK application that is
using an incompatible display protocol. Ordinarily, this doesn't happen,
because Clutter prefers its gdk backend, which works anyway; but MATE
forces it to use the x11 backend, so the bad situation can happen.

https://sources.debian.org/src/mate-session-manager/1.24.0-2/debian/default-settings/X11/Xsession.d/99mate-environment/
says MATE's override is a workaround for
https://bugs.launchpad.net/ubuntu/+source/parole/+bug/1462445 which
was a bug that existed in 2015. GNOME maintainers have not felt the need
to go back to the gdk backend, which suggests that the underlying bug
was either fixed at some point, or an application bug; the application
against which the bug was originally reported (parole) has since switched
away from Clutter altogether with the comment that its Clutter backend
was broken, which suggests that it might have been an application bug
after all. Please could someone confirm whether this workaround is
even necessary?

Some quick test results under GNOME 3.34, using totem as an example of
a GTK + Clutter app:

- CLUTTER_BACKEND=gdk GDK_BACKEND=x11 totem video.mp4: works
- CLUTTER_BACKEND=gdk GDK_BACKEND=wayland totem video.mp4: works
- CLUTTER_BACKEND=x11 GDK_BACKEND=x11 totem video.mp4: works
- CLUTTER_BACKEND=x11 GDK_BACKEND=wayland totem video.mp4: fails with
  "Unsupported backend", similar to this bug

I would expect that the two cases that do not force Wayland should also
work in MATE. If they don't, that's a Clutter bug which should be reported
separately.

If this workaround is no longer needed, I think the cleanest solution
would probably be for MATE to stop doing this?

Or if the workaround is still needed for whatever reason, then the
next best thing would be for MATE to use a bit less force. Judging by
the source code, CLUTTER_BACKEND="x11,*" is meant to prefer the x11
backend if possible, but fall back to any other available backend if
x11 isn't possible. Unfortunately, that doesn't seem to work: perhaps
there's a Clutter bug that is stopping the fallback between backends
from happening, or perhaps this mechanism is only designed to cope with
backends being completely absent, and not to cope with backends that are
not applicable to the current environment?

I would be reluctant to make the Clutter library disregard CLUTTER_BACKEND
completely, because that variable is meant to be a debugging/development
mechanism to get it onto a controlled code path.

I also don't think it's gnome-session's place to be unsetting
CLUTTER_BACKEND during session startup: it is not in a position to
tell the difference between undoing MATE's workaround (maybe good)
and undoing user/sysadmin configuration (bad).

    smcv