Bug#1148217: vino: numpad digits dead over VNC with modern xkeyboard-config KEYPAD type

[Luca]

Package: vino
Version: 3.22.0-6+numlock1
Severity: normal
Tags: patch

Since upgrading to trixie, numpad digit keys (KP_0-KP_9) produce nothing
at all over VNC, with any client (RealVNC, TigerVNC, Remmina). The
navigation keysyms work when NumLock is off, and / * - + . Enter work,
but digits never reach the X server. This is the same issue as Ubuntu
bug #2067259 (open since 22.04, where it was misdiagnosed as a NumLock
state desync).

Root cause (verified on trixie, vino 3.22.0-6):

vino_input_initialize_keycodes_xkb() builds its keysym->keycode table by
mapping vino's "shift level" to a key type's level via
mods->mask == ShiftMask. But the "KEYPAD" type in modern xkeyboard-config
(since 2.35, when map[Shift] = Level2 was removed from types/numpad) has
only a single map entry: map[NumLock] = Level2. With no Shift entry,
vino's levelmap for keypad types maps the "shift" level back to level 0,
so the scan only ever sees the navigation keysyms (KP_End etc.). No
bindings are ever created for KP_0-KP_9, and vino_input_fake_keypress()
silently drops those keysyms.

Verified by rebuilding with VINO_SERVER_DEBUG=input: "Got key press for
KP_1" is followed by no injection whatsoever, and the keybinding table
dump contains KP_End but no KP_1.

The attached patch (against 3.22.0, applies on top of the -6 patch
series) does two things:

1. When a key type has no Shift entry but has a NumLock entry, use the
   NumLock entry's level as the "shift" level. This restores bindings
   for the keypad digits. On old keymaps that have both Shift and
   NumLock entries pointing at the same level, behavior is unchanged.

2. Only fake NumLock press/release toggles for keypad bindings. NumLock
   does not affect the keysym produced by non-keypad keys, but vino was
   toggling it around *every* keypress whenever NumLock was on (the
   "Num.Lock(off) b Num.Lock(on)" per keystroke seen with screenkey in
   the Ubuntu report).

Tested live on trixie/MATE: digits inject correctly with NumLock on,
navigation keys with it off, the client/server NumLock desync case still
works via the existing fake-modifier path, and plain letters no longer
trigger NumLock flicker.


-- System Information:
Debian Release: 13.6
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 6.12.95+deb13-rt-amd64 (SMP w/4 CPU threads; PREEMPT)
Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages vino depends on:
ii  dconf-gsettings-backend [gsettings-backend]  0.40.0-5
ii  libavahi-client3                             0.8-16
ii  libavahi-common3                             0.8-16
ii  libavahi-glib1                               0.8-16
ii  libc6                                        2.41-12+deb13u3
ii  libcairo2                                    1.18.4-1+b1
ii  libgcrypt20                                  1.11.0-7+deb13u1
ii  libglib2.0-0t64                              2.84.4-3~deb13u3
ii  libgnutls30t64                               3.8.9-3+deb13u4
ii  libgtk-3-0t64                                3.24.49-3
ii  libice6                                      2:1.1.1-1
ii  libjpeg62-turbo                              1:2.1.5-4
ii  libnotify4                                   0.8.6-1
ii  libsecret-1-0                                0.21.7-1
ii  libsm6                                       2:1.2.6-1
ii  libx11-6                                     2:1.8.12-1
ii  libxdamage1                                  1:1.1.6-1+b2
ii  libxext6                                     2:1.3.4-1+b3
ii  libxfixes3                                   1:6.0.0-2+b4
ii  libxtst6                                     2:1.2.5-1
ii  zlib1g                                       1:1.3.dfsg+really1.3.1-1+b1

Versions of packages vino recommends:
ii  gvfs  1.57.2-2+deb13u1

Versions of packages vino suggests:
pn  gnome-control-center  <none>

-- no debconf information
-------------- next part --------------
--- a/server/vino-input.c
+++ b/server/vino-input.c
@@ -535,6 +535,9 @@
 
   global_input_data.alt_gr_keysym = XK_ISO_Level3_Shift;
 
+  /* Needed in the levelmap loop below */
+  global_input_data.num_lock_mod = XkbKeysymToModifiers (xdisplay, XK_Num_Lock);
+
   xkb = XkbGetMap (xdisplay, XkbAllClientInfoMask, XkbUseCoreKbd);
   g_assert (xkb != NULL);
 
@@ -565,6 +568,10 @@
 	  mods = &type->map[ktl].mods;
 	  if (mods->mask == ShiftMask)
 	    levelmap[kt][VINO_LEVEL_SHIFT] = type->map[ktl].level;
+	  else if (mods->mask == global_input_data.num_lock_mod &&
+		   levelmap[kt][VINO_LEVEL_SHIFT] == -1)
+	    /* keypad keys reach level 2 via NumLock, not Shift */
+	    levelmap[kt][VINO_LEVEL_SHIFT] = type->map[ktl].level;
 	  else if (mods->mask == LevelThreeMask)
 	    levelmap[kt][VINO_LEVEL_ALTGR] = type->map[ktl].level;
 	  else if (mods->mask == (ShiftMask | LevelThreeMask))
@@ -667,8 +674,7 @@
 	}
     }
 
-  /* Find NumLock modifier mask and get initial NumLock state */
-  global_input_data.num_lock_mod = XkbKeysymToModifiers (xdisplay, XK_Num_Lock);
+  /* Get initial NumLock state */
   if (XkbGetState (xdisplay, XkbUseCoreKbd, &state) == Success)
     {
       if (state.locked_mods & global_input_data.num_lock_mod)
@@ -964,7 +970,9 @@
 			 CurrentTime);
     }
 
-  if (VINO_LEVEL_IS_NUM_LOCK (cur_level) != VINO_LEVEL_IS_NUM_LOCK (binding->level))
+  /* NumLock only affects keypad keys */
+  if (binding->keypad &&
+      VINO_LEVEL_IS_NUM_LOCK (cur_level) != VINO_LEVEL_IS_NUM_LOCK (binding->level))
     {
       dprintf (INPUT, "Faking NumLock press/release\n");