Bug#918764: udev: "udevadm control --reload-rules" kills all processes except init

[Axel Beckert]

Control: found -1 239-15

Hi Michael,

Michael Biebl wrote:
> Am 12.01.19 um 01:02 schrieb Axel Beckert:
> > Control: reopen -1
> > Control: found -1 240-3
> 
> > *sigh* I'm sorry to say, but it just happened again with udev 240-3
> > and kernel 4.20-1~exp1.
> 
> Would be good to know, if it also happens with 239-15 or if it's caused
> by some other update.

I just downloaded udev and libudev1 239-15 from
https://snapshot.debian.org/, installed it and immediately afterwards
ran "udevadm control --reload-rules" and everything was gone again.
(Was running under kernel 4.20-1~exp1.)

After that I rebooted into 4.20-1~exp1 with the downgraded udev again,
ran "udevadm control --reload-rules" directly after reboot and nothing
(unexpected) happened.

Then I started to write this mail, doing not much more than logging in
on the text console (despite X was running) using zsh, running
ssh-agent, ssh-add and then ssh via autossh to connect to a screen
session on some other host to write the mail with mutt.

After about 20 minutes of uptime I was about to send that mail and
thought, I should just try running "udevadm control --reload-rules"
once again: And it killed all processes again -- systemd-udevd has
been started again, too:

~ # ps auxwww | fgrep udev
root     10696  0.0  0.0  13152  3592 ?        S    08:27   0:00 /lib/systemd/systemd-udevd
root     11582  0.0  0.0   8144   892 tty1     R+   08:33   0:00 grep -F --color=auto udev
~ # uptime
 08:33:49 up 25 min,  2 users,  load average: 0.07, 0.15, 0.34

Unfortunately there is not much in the syslog (had to start rsyslog
first again, too):

Jan 15 08:37:16 c6 kernel: [ 1717.930890] printk: systemd-udevd: 159 output lines suppressed due to ratelimiting

I though didn't get rsyslog to drop the rate-limiting and I'm a little
bit in a hurry at the moment.

Will report back later.

I must admit that I also had one crash/process killing yesterday where
I can't say what triggered it. aptitude just finished starting up in
TUI mode (inside screen started via ssh from remote) and I was
starting to browse through the package list while the connection
suddenly was lost (likely due to a killed sshd).

Some other facts gathered recently:

* With udev 239-15 the bootup lag is gone even without the "sleep 5".

* The "sleep 5" helped on another box (EeePC 900A with sysvinit) where
  drivers weren't loaded anymore and had to be specified manually in
  /etc/modules until the "sleep 5" was added.
* memtest86 and memtest86+ just show an empty screen. Will try again
  with grub's graphical mode disabled just to make sure the issue is
  not triggered by some memory fault. Question would be then why I
  could (within a reboot where it happened) reliably reproduce the
  issue again and again. Will report any findings on this front.
* As alternative I ran memtester for one night, no issues found. (Not
  sure if it was able to test everything as the affected box has 64 GB
  of RAM.)
* If the issue happens while using X, there's no chance to switch back
  to the text console with the getty to login again. The machine needs
  a hard reboot via reset or power button then.

> Tbh, udevd or udevadm control --reload killing all processes, sounds
> pretty strange.

Definitely.

> Can you also try to run udevd in debug mode to get a log from udevd (see
> /etc/udev/udev.conf) and also an strace of the udevadm command.

I think I alread sent the strace, but forgot the debug mode. Enabled
that when starting to write this mail, but it's currently caught by
rsyslogd's rate-limiting, see above.

		Regards, Axel
-- 
 ,''`.  |  Axel Beckert <abe at debian.org>, https://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-    |  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE