Bug#1148582: mutter: GPU memory leak while session is locked with external monitor in power saving, ends in system-wide OOM (upstream #4101, fixed in 50.5)

[Philippe Baret]

Package: libmutter-16-0
Version: 48.7-0+deb13u1
Severity: important
Tags: upstream fixed-upstream
Forwarded: https://gitlab.gnome.org/GNOME/mutter/-/issues/4101

Dear maintainers,

On trixie, gnome-shell leaks GPU buffers while the session is locked and
the external monitor is in power saving. GTT fills to its ceiling,
overflows into unmapped shmem and swap, and the kernel OOM killer then
kills small unrelated processes, which frees nothing. Four episodes
here in six days. This matches upstream issue 4101. Please consider
backporting the fix to trixie.

System: ThinkPad P14s Gen 2a, Ryzen 7 PRO 5850U (integrated Radeon),
32 GiB, kernel 6.12.86+deb13-amd64, mesa 25.0.7-2+deb13u1, gnome-shell
48.7-0+deb13u2, mutter 48.7-0+deb13u1, Wayland. Internal panel plus
ASUS PA278QV connected via USB-C dock, DisplayPort. 

Evidence (mem_info_gtt_used, /proc/meminfo, fdinfo drm-memory-gtt):

  2026-09-20 22:10: gtt_used 16183771136 of 16208060416, gnome-shell
  alone 15362 MiB, Shmem 12.2 GiB with /dev/shm and memfds empty,
  MemAvailable 64 MB. OOM kills 21:59 to 22:37. At about 22:37 all of
  it was released at once, same gnome-shell PID.

  Reproduction, 2026-09-21: Super+L, monitor sleeps, 15 minutes.
  gtt_used 240 MiB before, 5299834880 while locked, 1611337728 five
  minutes after unlock.

  Control: monitor switched off at its power button, then the same
  lock. gtt_used flat at about 530 MB. No leak.

Fix: https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/5285
("renderer/native: Don't keep around detached onscreens in power
saving"), merged 2026-09-03, in gnome-50 via !5325, released in 50.5.
Launchpad bug 2167374 says the backport needs two companion commits
(83c465ff, c145106d) and should not be taken without them. I have not
verified those hashes.

Workaround: switch the external monitor off before locking.

Thanks,
Philippe Baret

-- System Information:
Debian Release: 13.7
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 6.12.86+deb13-amd64 (SMP w/16 CPU threads; PREEMPT)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE
not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages libmutter-16-0 depends on:
ii  adwaita-icon-theme         48.1-1
ii  gsettings-desktop-schemas  48.0-1
ii  libatk1.0-0t64             2.56.2-1+deb13u2
ii  libc6                      2.41-12+deb13u4
ii  libcairo2                  1.18.4-1+b1
ii  libcanberra0               0.30-18
ii  libcolord2                 1.4.7-3
ii  libdisplay-info2           0.2.0-2
ii  libdrm2                    2.4.124-2
ii  libegl1                    1.7.0-1+b2
ii  libeis1                    1.3.901-1
ii  libfribidi0                1.0.16-1
ii  libgbm1                    25.0.7-2+deb13u1
ii  libgdk-pixbuf-2.0-0        2.42.12+dfsg-4+deb13u1
ii  libgl1                     1.7.0-1+b2
ii  libglib2.0-0t64            2.84.4-3~deb13u5
ii  libgnome-desktop-4-2t64    44.3-3
ii  libgraphene-1.0-0          1.10.8-5
ii  libgudev-1.0-0             238-6
ii  libharfbuzz0b              10.2.0-1+deb13u1
ii  libice6                    2:1.1.1-1
ii  libinput10                 1.28.1-1+deb13u1
ii  liblcms2-2                 2.16-2+deb13u2
ii  libpango-1.0-0             1.56.3-1
ii  libpangocairo-1.0-0        1.56.3-1
ii  libpipewire-0.3-0t64       1.4.2-1
ii  libpixman-1-0              0.44.0-3
ii  libsm6                     2:1.2.6-1
ii  libstartup-notification0   0.12-8
ii  libsystemd0                257.13-1~deb13u1
ii  libudev1                   257.13-1~deb13u1
ii  libwacom9                  2.14.0-1
ii  libwayland-server0         1.23.1-3
ii  libx11-6                   2:1.8.12-1
ii  libx11-xcb1                2:1.8.12-1
ii  libxau6                    1:1.0.11-1
ii  libxcb-randr0              1.17.0-2+b1
ii  libxcb-res0                1.17.0-2+b1
ii  libxcb1                    1.17.0-2+b1
ii  libxcomposite1             1:0.4.6-1
ii  libxcursor1                1:1.2.3-1
ii  libxdamage1                1:1.1.6-1+b2
ii  libxext6                   2:1.3.4-1+b3
ii  libxfixes3                 1:6.0.0-2+b4
ii  libxi6                     2:1.8.2-1
ii  libxinerama1               2:1.1.4-3+b4
ii  libxkbcommon-x11-0         1.7.0-2
ii  libxkbcommon0              1.7.0-2
ii  libxkbfile1                1:1.1.0-1+b4
ii  libxrandr2                 2:1.5.4-1+b3
ii  libxtst6                   2:1.2.5-1
ii  mutter-common              48.7-0+deb13u1
ii  mutter-common-bin          48.7-0+deb13u1

libmutter-16-0 recommends no packages.

libmutter-16-0 suggests no packages.

Versions of packages libmutter-16-0 is related to:
ii  libegl-mesa0 [libegl-vendor]  25.0.7-2+deb13u1
ii  libgl1-mesa-dri               25.0.7-2+deb13u1
ii  libglx-mesa0 [libglx-vendor]  25.0.7-2+deb13u1

-- no debconf information<br/>