Bug#831894: systemd: "Out of memory" timing error when calling systemctl in pipeline

[Brian Kroth]

Package: systemd
Version: 215-17+deb8u4
Severity: normal

Dear Maintainer,

Hi, I was trying to do some simple systemctl scripting for some service change rollouts and discovered what looks to be a timing bug related to calling systemctl in a pipeline.

When called like this we get an "Out of memory." error on most of our VMs, though they certainly have enough free memory available on the system.

I have not yet observed the issue on a physical machine yet.

# systemctl list-unit-files | awk '( $1 ~ /inetd.service$/ ) { print $1 }' | xargs -r -t systemctl stop
systemctl stop inetd.service openbsd-inetd.service 
Out of memory.


If on the other hand I do the same thing with either a slightly different (and less heavy weight) systemctl call at the beginning, or else add a short splay before the xargs call, then it works without the error:

# systemctl list-units | awk '( $1 ~ /inetd.service$/ ) { print $1 }' | xargs -r -t systemctl stop
systemctl stop inetd.service 
(no memory error reported)

# systemctl list-unit-files | awk '( $1 ~ /inetd.service$/ ) { print $1 }' | ( sleep .05; xargs -r -t systemctl stop )
systemctl stop inetd.service openbsd-inetd.service
(no memory error reported)


In case it helps, below is the output of running the first command with the SYSTEMD_LOG_LEVEL=debug environment variable set.

I've tried to strace it to see where exactly the error turns up, but that seems to disrupt the timing enough that the error message does not appear.

Let me know if you need any other info.

Thanks,
Brian


# systemctl list-unit-files | awk '( $1 ~ /inetd.service$/ ) { print $1 }' | SYSTEMD_LOG_LEVEL=debug xargs -r -t systemctl stop
systemctl stop inetd.service openbsd-inetd.service 
Calling manager for StopUnit on inetd.service, replace
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 object=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=StopUnit cookie=1 reply_cookie=0 error=n/a
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 object=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=GetUnit cookie=2 reply_cookie=0 error=n/a
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 object=/org/freedesktop/systemd1/unit/inetd_2eservice interface=org.freedesktop.DBus.Properties member=Get cookie=3 reply_cookie=0 error=n/a
Adding /org/freedesktop/systemd1/job/8238541 to the set
Calling manager for StopUnit on openbsd-inetd.service, replace
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 object=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=StopUnit cookie=4 reply_cookie=0 error=n/a
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 object=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=GetUnit cookie=5 reply_cookie=0 error=n/a
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 object=/org/freedesktop/systemd1/unit/inetd_2eservice interface=org.freedesktop.DBus.Properties member=Get cookie=6 reply_cookie=0 error=n/a
Adding /org/freedesktop/systemd1/job/8238541 to the set
Out of memory.
Got message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=UnitNew cookie=2 reply_cookie=0 error=n/a
Got D-Bus request: org.freedesktop.systemd1.Manager.UnitNew() on /org/freedesktop/systemd1
Got message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=JobNew cookie=3 reply_cookie=0 error=n/a
Got D-Bus request: org.freedesktop.systemd1.Manager.JobNew() on /org/freedesktop/systemd1
Got message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1/unit/inetd_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=7 reply_cookie=0 error=n/a
Got D-Bus request: org.freedesktop.DBus.Properties.PropertiesChanged() on /org/freedesktop/systemd1/unit/inetd_2eservice
Got message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1/unit/inetd_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=8 reply_cookie=0 error=n/a
Got D-Bus request: org.freedesktop.DBus.Properties.PropertiesChanged() on /org/freedesktop/systemd1/unit/inetd_2eservice
Got message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1/job/8238541 interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=9 reply_cookie=0 error=n/a
Got D-Bus request: org.freedesktop.DBus.Properties.PropertiesChanged() on /org/freedesktop/systemd1/job/8238541
Got message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=JobRemoved cookie=12 reply_cookie=0 error=n/a
Got D-Bus request: org.freedesktop.systemd1.Manager.JobRemoved() on /org/freedesktop/systemd1
Got result done/Success for job inetd.service
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 object=/org/freedesktop/systemd1/unit/inetd_2eservice interface=org.freedesktop.DBus.Properties member=Get cookie=7 reply_cookie=0 error=n/a
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 object=/org/freedesktop/systemd1/unit/inetd_2eservice interface=org.freedesktop.DBus.Properties member=Get cookie=8 reply_cookie=0 error=n/a
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 object=/org/freedesktop/systemd1/unit/openbsd_2dinetd_2eservice interface=org.freedesktop.DBus.Properties member=Get cookie=9 reply_cookie=0 error=n/a
Sent message type=method_call sender=n/a destination=org.freedesktop.systemd1 object=/org/freedesktop/systemd1/unit/openbsd_2dinetd_2eservice interface=org.freedesktop.DBus.Properties member=Get cookie=10 reply_cookie=0 error=n/a


-- Package-specific info:

-- System Information:
Debian Release: 8.5
  APT prefers stable
  APT policy: (500, 'stable'), (120, 'testing'), (110, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.5.0-0.bpo.2-amd64 (SMP w/1 CPU core)
Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages systemd depends on:
ii  acl             2.2.52-2
ii  adduser         3.113+nmu3
ii  initscripts     2.88dsf-59
ii  libacl1         2.2.52-2
ii  libaudit1       1:2.4-1+b1
ii  libblkid1       2.25.2-6
ii  libc6           2.19-18+deb8u4
ii  libcap2         1:2.24-8
ii  libcap2-bin     1:2.24-8
ii  libcryptsetup4  2:1.6.6-5
ii  libgcrypt20     1.6.3-2+deb8u1
ii  libkmod2        18-3
ii  liblzma5        5.1.1alpha+20120614-2+b3
ii  libpam0g        1.1.8-3.1+deb8u1+b1
ii  libselinux1     2.3-2
ii  libsystemd0     215-17+deb8u4
ii  mount           2.25.2-6
ii  sysv-rc         2.88dsf-59
ii  udev            215-17+deb8u4
ii  util-linux      2.25.2-6

Versions of packages systemd recommends:
ii  dbus            1.8.20-0+deb8u1
ii  libpam-systemd  215-17+deb8u4

Versions of packages systemd suggests:
pn  systemd-ui  <none>

-- Configuration Files:
/etc/systemd/journald.conf changed [not included]
/etc/systemd/logind.conf changed [not included]
/etc/systemd/system.conf changed [not included]

-- no debconf information