Bug#870462: gdm3: Can't login with gdm3 - "VT_ACTIVATE failed: Operation not permitted"

[Itai Shaked]

Package: gdm3
Version: 3.22.3-4
Severity: important

Dear Maintainer,

For a few months I am unable to login using gdm3. The greeter shows up
normally, but upon entering login information the screen flickers and the
system becomes barely-responsive for a few minutes, until I am left with a bare
TTY login prompt - this appears to be the result of gnome/gdm3 frantically
retrying and failing to start.

The X.org log contains the following info:

    (WW) xf86OpenConsole: VT_ACTIVATE failed: Operation not permitted
    (EE) Fatal server error:
    (EE) xf86OpenConsole: Switching VT failed

However, using lightdm I can login fine, so this does not seem to be an X.org
or driver-related problem, but rather something about gdm3.

Installing xserver-xorg-legacy does not solve this issue, however it did
produce a dmesg line about a segfault in libgtk-3-0.so. Reinstalling
`libgtk-3-0` got rid of the segfault, but still no luck - now with `xserver-
xorg-legacy` installed I'm getting the following in demsg:

    traps: gdm3[1318] general protection ip:564e456e74c2 sp:7ffc3c4017c0
error:0 in gdm3[564e456ce000+5d000]

Reinstalling gdm3 does not get rid of this message, and I am still unable to
login when using gdm - getting a constant flicker for about a minute until it
finally gives up. At the same time, lightdm manages to log in without any
problem.

The system has an integrated Intel graphics (Ivybridge / Intel HD4000), and no
other graphics adapter installed. I'm using the KMS driver.

The problem only occured after the update to 3.22.3-2, so this does not seem to
be a duplicate of bug #857995



-- System Information:
Debian Release: buster/sid
  APT prefers testing
  APT policy: (990, 'testing'), (700, 'unstable'), (500, 'testing-debug')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.11.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages gdm3 depends on:
ii  accountsservice                              0.6.43-1
ii  adduser                                      3.115
ii  dconf-cli                                    0.26.0-2+b1
ii  dconf-gsettings-backend                      0.26.0-2+b1
ii  debconf                                      1.5.63
ii  gir1.2-gdm-1.0                               3.22.3-4
ii  gnome-session [x-session-manager]            3.22.3-1
ii  gnome-session-bin                            3.22.3-1
ii  gnome-session-flashback [x-session-manager]  3.22.0-3
ii  gnome-settings-daemon                        3.22.2-5
ii  gnome-shell                                  3.22.3-3
ii  gnome-terminal [x-terminal-emulator]         3.22.2-1
ii  gsettings-desktop-schemas                    3.22.0-1
ii  libaccountsservice0                          0.6.43-1
ii  libaudit1                                    1:2.7.7-1+b2
ii  libc6                                        2.24-12
ii  libcanberra-gtk3-0                           0.30-3
ii  libcanberra0                                 0.30-3
ii  libgdk-pixbuf2.0-0                           2.36.5-2
ii  libgdm1                                      3.22.3-4
ii  libglib2.0-0                                 2.52.3-1
ii  libglib2.0-bin                               2.52.3-1
ii  libgtk-3-0                                   3.22.16-1
ii  libkeyutils1                                 1.5.9-9
ii  libpam-modules                               1.1.8-3.6
ii  libpam-runtime                               1.1.8-3.6
ii  libpam-systemd                               234-2
ii  libpam0g                                     1.1.8-3.6
ii  librsvg2-common                              2.40.18-1
ii  libselinux1                                  2.6-3+b2
ii  libsystemd0                                  234-2
ii  libwrap0                                     7.6.q-26
ii  libx11-6                                     2:1.6.4-3
ii  libxau6                                      1:1.0.8-1+b2
ii  libxcb1                                      1.12-1
ii  libxdmcp6                                    1:1.1.2-3
ii  lsb-base                                     9.20161125
ii  metacity [x-window-manager]                  1:3.24.1-1
ii  mutter [x-window-manager]                    3.22.4-2
ii  policykit-1                                  0.105-18
ii  twm [x-window-manager]                       1:1.0.9-1+b1
ii  ucf                                          3.0036
ii  x11-common                                   1:7.7+19
ii  x11-xserver-utils                            7.7+7+b1
ii  xfce4-session [x-session-manager]            4.12.1-5
ii  xfce4-terminal [x-terminal-emulator]         0.8.5.1-1
ii  xfwm4 [x-window-manager]                     4.12.4-1
ii  xterm [x-terminal-emulator]                  330-1

Versions of packages gdm3 recommends:
ii  at-spi2-core    2.24.1-1
ii  desktop-base    9.0.5
ii  x11-xkb-utils   7.7+3+b1
ii  xserver-xephyr  2:1.19.3-2
ii  xserver-xorg    1:7.7+19
ii  zenity          3.22.0-1+b1

Versions of packages gdm3 suggests:
ii  gnome-orca            3.24.0-1
ii  libpam-gnome-keyring  3.20.1-1

-- Configuration Files:
/etc/gdm3/PreSession/Default changed [not included]
/etc/gdm3/daemon.conf changed [not included]

-- debconf information:
* shared/default-x-display-manager: lightdm
  gdm3/daemon_name: /usr/sbin/gdm3