Bug#757213: systemd becomes non-responsive, connection refused to /run/systemd/private

Wed Aug 6 10:02:55 BST 2014

Package: systemd
Version: 208-6
Severity: important

Dear Maintainer,

I have two machines here with systemd becoming non-responsive after roughly one
to a few days of uptime. Both share the same hardware and software install.
This has been so for about a week (i.e. the problem reappears after a reboot).

The machines have been idle with no one logging in since the last reboot (which was yesterday), and this morning I observe the following:

root at potee:~# systemctl status
Failed to get D-Bus connection: Failed to connect to socket /run/systemd/private: Connection refused

As a consequence, many things don't work as they should, of course.

So far, one of these two machines has not  yet been rebooted.

Here is the last relevant excerpt from /var/log/syslog before the point systemd freezes. Note that the freeze occurs during normal operation, not during normal operation, not during a system upgrade.

Aug  6 07:04:24 potee systemd[1]: Starting CUPS Printing Service...
Aug  6 07:04:24 potee systemd[1]: Started CUPS Printing Service.
Aug  6 07:04:25 potee colord: Profile added: a210-Gray..
Aug  6 07:04:25 potee colord: Profile added: a210-CMYK..
Aug  6 07:04:25 potee colord[930]: (colord:930): Cd-WARNING **: failed to get session [pid 29125]: Unknown error -2
Aug  6 07:04:25 potee colord: Device added: cups-a210
Aug  6 07:04:56 potee colord: Automatic remove of a210-Gray.. from cups-a210
Aug  6 07:04:56 potee colord: Profile removed: a210-Gray..
Aug  6 07:04:56 potee colord: Automatic remove of a210-CMYK.. from cups-a210
Aug  6 07:04:56 potee colord: Profile removed: a210-CMYK..
Aug  6 07:04:56 potee colord: device removed: cups-a210
Aug  6 07:05:01 potee /USR/SBIN/CRON[29133]: (root) CMD (if [ -x /etc/munin/plugins/apt_all ]; then munin-run apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then munin-run apt update 7200 12 >/dev/null; fi)
Aug  6 07:05:24 potee systemd[1]: Starting CUPS Printing Service...
Aug  6 07:05:24 potee systemd[1]: Started CUPS Printing Service.
Aug  6 07:05:25 potee colord: Profile added: a210-Gray..
Aug  6 07:05:25 potee colord: Profile added: a210-CMYK..
Aug  6 07:05:25 potee colord[930]: (colord:930): Cd-WARNING **: failed to get session [pid 29134]: Unknown error -2
Aug  6 07:05:25 potee colord: Device added: cups-a210
Aug  6 07:05:56 potee colord: Automatic remove of a210-Gray.. from cups-a210
Aug  6 07:05:56 potee colord: Profile removed: a210-Gray..
Aug  6 07:05:56 potee colord: Automatic remove of a210-CMYK.. from cups-a210
Aug  6 07:05:56 potee colord: Profile removed: a210-CMYK..
Aug  6 07:05:56 potee colord: device removed: cups-a210
Aug  6 07:05:56 potee systemd[1]: Assertion 'close_nointr(fd) == 0' failed at ../src/shared/util.c:188, function safe_close(). Aborting.
Aug  6 07:05:56 potee systemd[1]: Caught <ABRT>, dumped core as pid 29139.
Aug  6 07:05:56 potee systemd[1]: Freezing execution.

The second machine freezes because of the same failed assertion, after similar syslog activity.

There's obviously something fishy with how CUPS behaves here, which might be caused by local misconfiguration or perhaps a CUPS bug, I'll investigate that separately. However systemd should not freeze like this.

[I've looked at #742322; it's probably not totally unrelated, but as far as I can tell there is no dead symlink at stake here].

Given the fact that the situation seems somewhat reproducible if one is ready
to wait a few hours, I would says that instrumenting the code to gather more
info from the point of failure can be envisioned if needed.

I'm happy to help with further information.

Best regards,

E.

-- Package-specific info:

-- System Information:
Debian Release: jessie/sid
  APT prefers testing
  APT policy: (900, 'testing'), (100, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.14-2-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages systemd depends on:
ii  acl                  2.2.52-1
ii  adduser              3.113+nmu3
ii  initscripts          2.88dsf-53.2
ii  libacl1              2.2.52-1
ii  libaudit1            1:2.3.7-1
ii  libblkid1            2.20.1-5.8
ii  libc6                2.19-7
ii  libcap2              1:2.24-3
ii  libcap2-bin          1:2.24-3
ii  libcryptsetup4       2:1.6.4-4
ii  libdbus-1-3          1.8.6-1
ii  libgcrypt11          1.5.3-5
ii  libkmod2             18-1
ii  liblzma5             5.1.1alpha+20120614-2
ii  libpam0g             1.1.8-3
ii  libselinux1          2.3-1
ii  libsystemd-daemon0   208-6
ii  libsystemd-journal0  208-6
ii  libsystemd-login0    208-6
ii  libudev1             208-6
ii  libwrap0             7.6.q-25
ii  sysv-rc              2.88dsf-53.2
ii  udev                 208-6
ii  util-linux           2.20.1-5.8

Versions of packages systemd recommends:
ii  libpam-systemd  208-6

Versions of packages systemd suggests:
pn  systemd-ui  <none>

-- no debconf information
-------------- next part --------------
[OVERRIDDEN] /etc/modules-load.d/ipmievd.conf -> /usr/lib/modules-load.d/ipmievd.conf

Files /usr/lib/modules-load.d/ipmievd.conf and /etc/modules-load.d/ipmievd.conf are identical

[EXTENDED]   /lib/systemd/system/cups.socket -> /etc/systemd/system/cups.socket.d/cupsd-listen.conf

2 overridden configuration files found.
-------------- next part --------------
==> /var/lib/systemd/deb-systemd-helper-enabled/cron.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/cron.service

==> /var/lib/systemd/deb-systemd-helper-enabled/virtlockd.socket.dsh-also <==
/etc/systemd/system/sockets.target.wants/virtlockd.socket

==> /var/lib/systemd/deb-systemd-helper-enabled/accounts-daemon.service.dsh-also <==
/etc/systemd/system/graphical.target.wants/accounts-daemon.service

==> /var/lib/systemd/deb-systemd-helper-enabled/netfilter-persistent.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/netfilter-persistent.service

==> /var/lib/systemd/deb-systemd-helper-enabled/uptimed.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/uptimed.service

==> /var/lib/systemd/deb-systemd-helper-enabled/munin-node.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/munin-node.service

==> /var/lib/systemd/deb-systemd-helper-enabled/libvirt-bin.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/libvirt-bin.service

==> /var/lib/systemd/deb-systemd-helper-enabled/ssh.socket.dsh-also <==
/etc/systemd/system/sockets.target.wants/ssh.socket

==> /var/lib/systemd/deb-systemd-helper-enabled/ModemManager.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/ModemManager.service
/etc/systemd/system/dbus-org.freedesktop.ModemManager1.service

==> /var/lib/systemd/deb-systemd-helper-enabled/lvm2-activation-early.service.dsh-also <==
/etc/systemd/system/local-fs.target.wants/lvm2-activation-early.service

==> /var/lib/systemd/deb-systemd-helper-enabled/lvm2-activation.service.dsh-also <==
/etc/systemd/system/local-fs.target.wants/lvm2-activation.service

==> /var/lib/systemd/deb-systemd-helper-enabled/NetworkManager-dispatcher.service.dsh-also <==
/etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service

==> /var/lib/systemd/deb-systemd-helper-enabled/NetworkManager.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/NetworkManager.service
/etc/systemd/system/dbus-org.freedesktop.NetworkManager.service
/etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service

==> /var/lib/systemd/deb-systemd-helper-enabled/dbus-org.freedesktop.Avahi.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/cups.service.dsh-also <==
/etc/systemd/system/sockets.target.wants/cups.socket
/etc/systemd/system/paths.target.wants/cups.path
/etc/systemd/system/printer.target.wants/cups.service

==> /var/lib/systemd/deb-systemd-helper-enabled/local-fs.target.wants/lvm2-activation-early.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/local-fs.target.wants/lvm2-activation.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/ssh.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/ssh.service
/etc/systemd/system/sshd.service

==> /var/lib/systemd/deb-systemd-helper-enabled/pppd-dns.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/pppd-dns.service

==> /var/lib/systemd/deb-systemd-helper-enabled/printer.target.wants/cups.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/shutdown.target.wants/unattended-upgrades.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/cups.path.dsh-also <==
/etc/systemd/system/paths.target.wants/cups.path

==> /var/lib/systemd/deb-systemd-helper-enabled/lvm2-lvmetad.socket.dsh-also <==
/etc/systemd/system/sockets.target.wants/lvm2-lvmetad.socket

==> /var/lib/systemd/deb-systemd-helper-enabled/graphical.target.wants/accounts-daemon.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/bluetooth.service.dsh-also <==
/etc/systemd/system/bluetooth.target.wants/bluetooth.service
/etc/systemd/system/dbus-org.bluez.service

==> /var/lib/systemd/deb-systemd-helper-enabled/libvirt-guests.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/libvirt-guests.service

==> /var/lib/systemd/deb-systemd-helper-enabled/atd.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/atd.service

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/ssh.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/uptimed.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/libvirt-bin.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/cups.path <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/binfmt-support.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/munin-node.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/atd.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/pppd-dns.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/avahi-daemon.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/netfilter-persistent.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/libvirt-guests.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/ModemManager.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/cron.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/cups-browsed.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/rsyslog.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/anacron.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/multi-user.target.wants/NetworkManager.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/dbus-org.freedesktop.ModemManager1.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/syslog.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/unattended-upgrades.service.dsh-also <==
/etc/systemd/system/shutdown.target.wants/unattended-upgrades.service

==> /var/lib/systemd/deb-systemd-helper-enabled/sockets.target.wants/cups.socket <==

==> /var/lib/systemd/deb-systemd-helper-enabled/sockets.target.wants/lvm2-lvmetad.socket <==

==> /var/lib/systemd/deb-systemd-helper-enabled/sockets.target.wants/virtlockd.socket <==

==> /var/lib/systemd/deb-systemd-helper-enabled/sockets.target.wants/avahi-daemon.socket <==

==> /var/lib/systemd/deb-systemd-helper-enabled/cups-browsed.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/cups-browsed.service

==> /var/lib/systemd/deb-systemd-helper-enabled/dbus-org.freedesktop.nm-dispatcher.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/anacron.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/anacron.service

==> /var/lib/systemd/deb-systemd-helper-enabled/binfmt-support.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/binfmt-support.service

==> /var/lib/systemd/deb-systemd-helper-enabled/dbus-org.bluez.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/avahi-daemon.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/avahi-daemon.service
/etc/systemd/system/sockets.target.wants/avahi-daemon.socket
/etc/systemd/system/dbus-org.freedesktop.Avahi.service

==> /var/lib/systemd/deb-systemd-helper-enabled/virtlockd.service.dsh-also <==
/etc/systemd/system/sockets.target.wants/virtlockd.socket

==> /var/lib/systemd/deb-systemd-helper-enabled/avahi-daemon.socket.dsh-also <==
/etc/systemd/system/sockets.target.wants/avahi-daemon.socket

==> /var/lib/systemd/deb-systemd-helper-enabled/sshd.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/spamassassin.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/spamassassin.service

==> /var/lib/systemd/deb-systemd-helper-enabled/NetworkManager-wait-online.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/NetworkManager-wait-online.service

==> /var/lib/systemd/deb-systemd-helper-enabled/dbus-org.freedesktop.NetworkManager.service <==

==> /var/lib/systemd/deb-systemd-helper-enabled/rsyslog.service.dsh-also <==
/etc/systemd/system/multi-user.target.wants/rsyslog.service
/etc/systemd/system/syslog.service

==> /var/lib/systemd/deb-systemd-helper-enabled/cups.socket.dsh-also <==
/etc/systemd/system/sockets.target.wants/cups.socket

==> /var/lib/systemd/deb-systemd-helper-enabled/bluetooth.target.wants/bluetooth.service <==