Bug#816276: systemd: shutdown fails if swap can't be deactivated

Mon Feb 29 11:43:14 GMT 2016

Package: systemd
Version: 215-17+deb8u3
Severity: normal

Hello,

I'm running jessie inside a low-memory (k)vm, 384M of RAM plus
swap. This has been tuned so that as much as possible is swapped out
during normal operation but there's no thrashing, i.e. the working set
easily fits into RAM.

Shutting down this vm will fail more often than not because systemd
tries to disable swap very early in the process, before the main
daemon (logitechmediaserver) has shut down. The shutdown process will
then hang until the vm is force-killed by libvirt after about 5 mins
or so.

1) Why is swap deactivated first thing? IMHO it should stick around
until after all "normal" processes have exited / been killed. After
all, if swap can be deactivated in the running system it wasn't really
needed in the first place.

2) Why does a failure to deactivate swap result in the shutdown
process hanging indefinitely? Shouldn't systemd either retry or fail
gracefully (i.e., ignore the problem)?

(For this specific use case I'll also look into giving the vm more RAM
and having the host swap out more, but I do not know yet if parts of
vm memory can be efficiently swapped out host-side.)

Regards,
Christian Pernegger

# fstab entry for swap
/dev/vdb none swap sw 0 0

# log of failed shutdown

Feb 26 20:29:00 slimserver systemd-logind[410]: Power key pressed.
Feb 26 20:29:00 slimserver systemd-logind[410]: Powering Off...
Feb 26 20:29:00 slimserver systemd-logind[410]: System is powering down.
Feb 26 20:29:01 slimserver nullmailer[839]: Stopping mail-transfer-agent: nullmailer.
Feb 26 20:29:01 slimserver swapoff[838]: swapoff: /dev/vdb: swapoff failed: Nicht genügend Hauptspeicher verfügbar # that's "not enough memory [available]"
Feb 26 20:29:01 slimserver systemd[1]: dev-vdb.swap swap process exited, code=exited status=255
Feb 26 20:29:01 slimserver systemd[1]: Unit dev-vdb.swap entered failed state.
Feb 26 20:29:01 slimserver sshd[405]: Received signal 15; terminating.
Feb 26 20:29:01 slimserver kernel:  vdb: unknown partition table
Feb 26 20:29:02 slimserver logitechmediaserver[840]: Stopping Logitech Media Server.
Feb 26 20:29:02 slimserver hwclock[837]: hwclock from util-linux 2.25.2
Feb 26 20:29:02 slimserver hwclock[837]: Using the /dev interface to the clock.
Feb 26 20:29:02 slimserver hwclock[837]: Last drift adjustment done at 1456410669 seconds after 1969
Feb 26 20:29:02 slimserver hwclock[837]: Last calibration done at 1456410669 seconds after 1969
Feb 26 20:29:02 slimserver hwclock[837]: Hardware clock is on UTC time
Feb 26 20:29:02 slimserver hwclock[837]: Assuming hardware clock is kept in UTC time.
Feb 26 20:29:02 slimserver hwclock[837]: Waiting for clock tick...
Feb 26 20:29:02 slimserver hwclock[837]: ...got clock tick
Feb 26 20:29:02 slimserver hwclock[837]: Time read from Hardware Clock: 2016/02/26 19:29:01
Feb 26 20:29:02 slimserver hwclock[837]: Hw clock time : 2016/02/26 19:29:01 = 1456514941 seconds since 1969
Feb 26 20:29:02 slimserver hwclock[837]: 1456514942.500000 is close enough to 1456514942.500000 (0.000000 < 0.001000)
Feb 26 20:29:02 slimserver hwclock[837]: Set RTC to 1456514942 (1456514942 + 0; refsystime = 1456514942.000000)
Feb 26 20:29:02 slimserver hwclock[837]: Setting Hardware Clock to 19:29:02 = 1456514942 seconds since 1969
Feb 26 20:29:02 slimserver hwclock[837]: ioctl(RTC_SET_TIME) was successful.
Feb 26 20:29:02 slimserver hwclock[837]: Clock drifted 0.4 seconds in the past 104273 seconds in spite of a drift factor of 0.520768 seconds/day.
Feb 26 20:29:02 slimserver hwclock[837]: Adjusting drift factor by 0.319724 seconds/day
Feb 26 20:29:02 slimserver dhclient[895]: Killed old client process
Feb 26 20:29:03 slimserver networking[876]: Deconfiguring network interfaces...Killed old client process
Feb 26 20:29:04 slimserver dhclient[895]: Internet Systems Consortium DHCP Client 4.3.1
Feb 26 20:29:04 slimserver dhclient[895]: Copyright 2004-2014 Internet Systems Consortium.
Feb 26 20:29:04 slimserver dhclient[895]: All rights reserved.
Feb 26 20:29:04 slimserver dhclient[895]: For info, please visit https://www.isc.org/software/dhcp/
Feb 26 20:29:04 slimserver dhclient[895]:
Feb 26 20:29:04 slimserver dhclient[895]: Listening on LPF/eth0/52:54:00:21:14:bf
Feb 26 20:29:04 slimserver dhclient[895]: Sending on   LPF/eth0/52:54:00:21:14:bf
Feb 26 20:29:04 slimserver dhclient[895]: Sending on   Socket/fallback
Feb 26 20:29:04 slimserver networking[876]: Internet Systems Consortium DHCP Client 4.3.1
Feb 26 20:29:04 slimserver networking[876]: Copyright 2004-2014 Internet Systems Consortium.
Feb 26 20:29:04 slimserver networking[876]: All rights reserved.
Feb 26 20:29:04 slimserver networking[876]: For info, please visit https://www.isc.org/software/dhcp/
Feb 26 20:29:04 slimserver networking[876]: Listening on LPF/eth0/52:54:00:21:14:bf
Feb 26 20:29:04 slimserver networking[876]: Sending on   LPF/eth0/52:54:00:21:14:bf
Feb 26 20:29:04 slimserver networking[876]: Sending on   Socket/fallback
Feb 26 20:29:04 slimserver dhclient[895]: DHCPRELEASE on eth0 to 192.168.0.1 port 67
Feb 26 20:29:04 slimserver networking[876]: DHCPRELEASE on eth0 to 192.168.0.1 port 67
Feb 26 20:29:04 slimserver networking[876]: done.

# and that's it. At this point there's neither ssh nor console access available, so destroying the vm is the only option.

-- Package-specific info:

-- System Information:
Debian Release: 8.3
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.16.0-4-amd64 (SMP w/1 CPU core)
Locale: LANG=de_AT.UTF-8, LC_CTYPE=de_AT.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages systemd depends on:
ii  acl             2.2.52-2
ii  adduser         3.113+nmu3
ii  initscripts     2.88dsf-59
ii  libacl1         2.2.52-2
ii  libaudit1       1:2.4-1+b1
ii  libblkid1       2.25.2-6
ii  libc6           2.19-18+deb8u3
ii  libcap2         1:2.24-8
ii  libcap2-bin     1:2.24-8
ii  libcryptsetup4  2:1.6.6-5
ii  libgcrypt20     1.6.3-2+deb8u1
ii  libkmod2        18-3
ii  liblzma5        5.1.1alpha+20120614-2+b3
ii  libpam0g        1.1.8-3.1+deb8u1
ii  libselinux1     2.3-2
ii  libsystemd0     215-17+deb8u3
ii  mount           2.25.2-6
ii  sysv-rc         2.88dsf-59
ii  udev            215-17+deb8u3
ii  util-linux      2.25.2-6

Versions of packages systemd recommends:
ii  dbus            1.8.20-0+deb8u1
pn  libpam-systemd  <none>

Versions of packages systemd suggests:
pn  systemd-ui  <none>

-- Configuration Files:
/etc/systemd/journald.conf changed [not included]

-- no debconf information