Bug#800770: Boot fails with: Cogl-ERROR **: Failed to create texture 2d due to size/format constraints

Sat Oct 3 12:40:29 UTC 2015

Package: mutter
Version: 3.14.4-1~deb8u1
Severity: important

Dear Maintainer,

This fixes the nouveau black screen at boot cogl error bug on my Dell C840.
This causes the boot process to terminate with an error message in syslog:
gnome-session[778]: (gnome-shell:814):
Cogl-ERROR **: Failed to create texture 2d due to size/format constraints
Disabling modesetting allows boot to proceed but disables screen size detection
as a side effect, resulting in an undersized screen being displayed - about
2/3 normal size on my hardware.
The solution requires modifications to both mutter and gnome-shell.
cogl made some changes which meant apps should use the 2d functions or fall
back on 2d sliced functions, the fallback was missed out so it failed on
various hardware - on some very recent machines too, not just older ones.
>From cogl source package:
 * Cogl does not aim to be a lowest common denominator API, it aims to
 * expose all the interesting features of GPUs to application which
 * means applications have some responsibility to explicitly check
 * that certain features are available before depending on them.
These modifications simply take the functionality of deprecated cogl, e.g.:
cogl_texture_new_from_data ()
and use the new equivalents to provide the same thing:
cogl_texture_2d_new_from_data
cogl_texture_2d_sliced_new_from_data
The deprecated function manually wrote a mipmap which I have also copied, this
makes no difference to my hardware, it might for others so I left it in.
The changes to cogl apparently would allow fast moving graphic images to run
better on hardware that supports it. It's not especially relevant for Gnome.
If you were drawing lots of things then you could test to see if NPOT
textures were supported once, then use 2d_new or 2d_sliced as appropriate.
The deprecated function checked internally every time so was simpler to use.
When Gnome was updated it missed out the second path, but obviously worked on
the developers particular hardware so they didn't experience the bug.

But I still get an error after fixing the first bug:
gnome-session[779]: (gnome-shell:815): Cogl-WARNING **:
Your hardware does not have enough texture unitsto handle this many texture
layers

I would guess you can build up an onscreen image using different layers and
that here every element or letter of text is being put in a separate layer so
it's using them all up, although I didn't get to exploring this other bug.
The login screen is then missing some letters where there should be text.
Login works though and I can continue to the Gnome Flashback desktop, this
uses metacity not mutter, and it works normally.
So for my purposes it fixes the problem, and I think these modifications
would make it easier for anyone to take things further, it's hard fixing an
OS that boots to a black screen, although you could use a getty.
I think this is more or less how cogl intended their changes to be
implemented so it's hopefully a step in the right direction even if it
doesn't fix all the issues involving gnome and hardware accelerated graphics.
The number of texture units is found by a private function in cogl - this
might need to be exported as part of a solution.

Here's what I did to modify then install the packages - mutter won't work at
all in /usr/local, gnome-shell works in either /usr or /usr/local but gives
various extra error messages in syslog, probably something else should be
reset or configured for the installation, it works normally though AFAICT.
These are installed in the wrong place for one thing:
$ ls /usr/libexec
gnome-shell-calendar-server
gnome-shell-hotplug-sniffer
gnome-shell-perf-helper
gnome-shell-portal-helper
mutter-restart-helper

../configure allows different locations for the libexec directory - but there
is no libexec directory in the Debian gnome-shell binary package.
(There are probably easier or better ways to do the procedure below.)
-------------------------------------------------

Modify, compile and install gnome-shell, get the Debian source files:
gnome-shell_3.14.4-1~deb8u1.debian.tar.xz
gnome-shell_3.14.4-1~deb8u1.dsc
gnome-shell_3.14.4.orig.tar.xz

$ dpkg-source -x gnome-shell_3.14.4-1~deb8u1.dsc
Applies Debian patches and unpacks compressed archive.

$ dpkg-checkbuilddeps gnome-shell-3.14.4/debian/control
Then install the required build dependency packages.

Modify source, then configure (or configure then modify should work too)
..../gnome-shell-3.14.4$ ./configure --enable-compile-warnings=yes
--prefix=/usr  --enable-networkmanager --enable-systemd

$ make
# make install
# make uninstall (to remove, I only overwrote it with new versions instead)
(original gnome-shell would need to be reinstalled as it got overwritten,
if you want it back as it was.)

-------------------------------------------------

mutter, mostly as above:
mutter_3.14.4-1~deb8u1.debian.tar.xz
mutter_3.14.4-1~deb8u1.dsc
mutter_3.14.4.orig.tar.xz

$ dpkg-source -x mutter_3.14.4-1~deb8u1.dsc
$ dpkg-checkbuilddeps gnome-shell-3.14.4/debian/control

Avoid configure problem using:
$ gnome-autogen.sh --enable-compile-warnings=yes
--enable-startup-notification=yes --enable-debug --prefix=/usr

$ make
# make install

-------------------------------------------------

I will (if reportbug allows) attach the following files in a tar.gz archive:
cogl-info_output.txt  grub-vbeinfo-Dell_C840.jpg  testing-mutter.diff
final-testing-syslog  mutter.diff
gnome-shell.diff      testing-gnome-shell.diff

cogl-info_output.txt and grub-vbeinfo-Dell_C840.jpg show some details about
the hardware I am using and it's graphics capabilities.
testing-gnome-shell.diff and testing-mutter.diff are the code changes that
include developmental debugging output which can be seen in the syslog,
final-testing-syslog showing the altered code and how it is called at boot.
The final patches which I offer for your use (or amusement if I'm wrong!) are:
gnome-shell.diff
mutter.diff

The mutter patch adds the following make warnings, it works though as these
are some obscure void pointer thing that I never got to fixing:
backends/meta-cursor.c: In function 'meta_cursor_image_load_from_xcursor_image':

backends/meta-cursor.c:288:22: warning: assignment from incompatible pointer typ
e [enabled by default]
       image->texture = COGL_TEXTURE (cogl_texture_2d_sliced_new_from_data (cogl
_context,
                      ^
  CC       backends/meta-cursor-tracker.lo
backends/meta-cursor-tracker.c: In function 'ensure_xfixes_cursor':
backends/meta-cursor-tracker.c:299:14: warning: assignment from incompatible poi
nter type [enabled by default]
       sprite = COGL_TEXTURE (cogl_texture_2d_sliced_new_from_data (ctx,
              ^

I have been using this now for a couple of months without issues other than
those described here, it took a while to gather it all and put it in a
readable format and I've been doing other things too.
Best wishes,
www.cxperimental.weebly.com

-- System Information:
Debian Release: 8.1
  APT prefers stable
  APT policy: (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 3.16.0-4-686-pae (SMP w/1 CPU core)
Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages mutter depends on:
ii  gnome-themes-standard      3.14.2.2-1
ii  gsettings-desktop-schemas  3.14.1-1
ii  libc6                      2.19-18
ii  libclutter-1.0-0           1.20.0-1
ii  libglib2.0-0               2.42.1-1
ii  libmutter0e                3.14.4-1~deb8u1
ii  libx11-6                   2:1.6.2-3
ii  libxcomposite1             1:0.4.4-1
ii  mutter-common              3.14.4-1~deb8u1
ii  zenity                     3.14.0-1

Versions of packages mutter recommends:
ii  gnome-session [x-session-manager]            3.14.0-2
ii  gnome-session-flashback [x-session-manager]  3.8.1-7

Versions of packages mutter suggests:
ii  gnome-control-center  1:3.14.2-3
pn  xdg-user-dirs         <none>

-- no debconf information