[Pkg-libvirt-maintainers] Bug#905772: virtlogd dependency loop cuaing upgrade failures

Christian Ehrhardt christian.ehrhardt at canonical.com
Thu Aug 9 11:13:28 BST 2018


Package: libvirt
Version: 4.6.0-1
Severity: normal

Hi,
I ran into this while testing libvirt 4.6 and opened initially just Ubuntu
bug [1].
But I found nothing Ubuntu-special so I recreated on Debian and realized
that you are affected as well.

I have no solution yet, but would be happy if we can sync on ideas and
implement the same eventually whatever we come up with.
But first let me outline the issue

TL;DR:
- virtlogd.service has since 4.2 a requires statement to both sockets
(virtlods.socket and virtlogd-admin.socket)
- before an upgrade usually virtlogd.socket and virtlogd.service are running
- on the upgrade virtlogd-admin.socket gets installed
Note: all those services/sockets run with no-restart-on-upgrade
- virtlogd should not be restarted, so it calls systemctl reload
- dh_systemd_enable: the two sockets files get enabled
- dh_systemd_start: will only call deb-systemd-invoke start on the sockets
ignoring errors
- /etc/init.d/virtlogd exists, so the postinst calls "invoke-rc.d virtlogd
start"

Remember that virtlogd is already running and working fine.
But that last step makes it realize that the new requires dependency to
virtlogd-admin.socket is not fulfilled and aborts the installation.

In a similar fashion /etc/init.d/libvirtd always gives an extra potentially
fatal "start" action.
Since the latter is not instaklled with no-restart dh_installinit will even
make this a restart call on upgrades.

As a POC I took away:
  $ mv  /etc/init.d/virtlogd /etc/init.d/virtlogd.nothere
But then realized we still start libvirtd through "invoke-rc.d libvirtd
$_dh_action" which due to this line in the init script:
  # Required-Start:    $network $local_fs $remote_fs $syslog virtlogd
will fail to start now.
Overall this is preferred by dh_systemd_start since both scripts are
installed.

I think it might be time to drop both sysV scripts as it seems that would
resolve all of it by eliminating the double calls.
But I need to test that without messing in a container, but a real package
build to be sure.

Details of such an upgrade - example is stable 3.0 -> testing 4.5

PRE
root at debian-stretch:~# systemctl status virtlogd.service virtlogd.socket
virtlogd-admin.socket virtlockd.service virtlockd.socket
virtlockd-admin.socket --no-pager --lines 2
● virtlogd.service - Virtual machine log manager
   Loaded: loaded (/lib/systemd/system/virtlogd.service; indirect; vendor
preset: enabled)
   Active: active (running) since Thu 2018-08-09 09:28:33 UTC; 7s ago
     Docs: man:virtlogd(8)
           http://libvirt.org
 Main PID: 7080 (virtlogd)
    Tasks: 2 (limit: 4915)
   CGroup: /system.slice/virtlogd.service
           └─7080 /usr/sbin/virtlogd

Aug 09 09:28:35 debian-stretch systemd[1]: virtlogd.service: Failed to
reset devices.list: Operation not permitted
Aug 09 09:28:37 debian-stretch systemd[1]: virtlogd.service: Failed to
reset devices.list: Operation not permitted

● virtlogd.socket - Virtual machine log manager socket
   Loaded: loaded (/lib/systemd/system/virtlogd.socket; enabled; vendor
preset: enabled)
   Active: active (running) since Thu 2018-08-09 09:28:33 UTC; 7s ago
   Listen: /var/run/libvirt/virtlogd-sock (Stream)

Aug 09 09:28:33 debian-stretch systemd[1]: Listening on Virtual machine log
manager socket.
Unit virtlogd-admin.socket could not be found.

● virtlockd.service - Virtual machine lock manager
   Loaded: loaded (/lib/systemd/system/virtlockd.service; indirect; vendor
preset: enabled)
   Active: inactive (dead)
     Docs: man:virtlockd(8)
           http://libvirt.org

● virtlockd.socket - Virtual machine lock manager socket
   Loaded: loaded (/lib/systemd/system/virtlockd.socket; enabled; vendor
preset: enabled)
   Active: active (listening) since Thu 2018-08-09 09:28:34 UTC; 7s ago
   Listen: /var/run/libvirt/virtlockd-sock (Stream)

Aug 09 09:28:34 debian-stretch systemd[1]: Listening on Virtual machine
lock manager socket.
Unit virtlockd-admin.socket could not be found.





Upgrade just libvirt and dependencies - stretch to testing.

# apt install libvirt-daemon-system
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer
required:
  libboost-iostreams1.62.0 libboost-random1.62.0 libboost-system1.62.0
libboost-thread1.62.0 librados2 librbd1
Use 'apt autoremove' to remove them.
The following additional packages will be installed:
  ca-certificates libcap-ng0 libcom-err2 libcomerr2 libcurl3-gnutls
libidn2-0 libnghttp2-14 libpsl5 librtmp1 libunistring2 libvirt-clients
  libvirt-daemon libvirt0 openssl publicsuffix
Suggested packages:
  libvirt-daemon-driver-storage-gluster libvirt-daemon-driver-storage-rbd
libvirt-daemon-driver-storage-sheepdog
  libvirt-daemon-driver-storage-zfs numad apparmor auditd nfs-common
open-iscsi pm-utils radvd systemtap zfsutils
The following NEW packages will be installed:
  ca-certificates libcom-err2 libcurl3-gnutls libidn2-0 libnghttp2-14
libpsl5 librtmp1 libunistring2 openssl publicsuffix
The following packages will be upgraded:
  libcap-ng0 libcomerr2 libvirt-clients libvirt-daemon
libvirt-daemon-system libvirt0
6 upgraded, 10 newly installed, 0 to remove and 248 not upgraded.
Need to get 9437 kB of archives.
After this operation, 5147 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
[...]
Setting up libvirt-daemon-system (4.5.0-1) ...
Installing new version of config file
/etc/apparmor.d/abstractions/libvirt-qemu ...
Installing new version of config file /etc/apparmor.d/libvirt/TEMPLATE.lxc
...
Installing new version of config file /etc/apparmor.d/libvirt/TEMPLATE.qemu
...
Installing new version of config file
/etc/apparmor.d/usr.lib.libvirt.virt-aa-helper ...
Installing new version of config file /etc/apparmor.d/usr.sbin.libvirtd ...
Installing new version of config file /etc/default/libvirt-guests ...
Installing new version of config file /etc/init.d/libvirt-guests ...
Installing new version of config file /etc/libvirt/libvirtd.conf ...
Installing new version of config file /etc/libvirt/libxl.conf ...
Installing new version of config file /etc/libvirt/qemu.conf ...
Installing new version of config file /etc/libvirt/virtlockd.conf ...
Installing new version of config file /etc/libvirt/virtlogd.conf ...
Installing new version of config file /etc/logrotate.d/libvirtd.libxl ...
Installing new version of config file /etc/logrotate.d/libvirtd.lxc ...
Installing new version of config file /etc/logrotate.d/libvirtd.qemu ...
Installing new version of config file /etc/logrotate.d/libvirtd.uml ...
Installing new version of config file /etc/sasl2/libvirt.conf ...
Created symlink
/etc/systemd/system/sockets.target.wants/virtlockd-admin.socket →
/lib/systemd/system/virtlockd-admin.socket.
Created symlink
/etc/systemd/system/sockets.target.wants/virtlogd-admin.socket →
/lib/systemd/system/virtlogd-admin.socket.
virtlockd.service is a disabled or a static unit, not starting it.
Job for virtlogd-admin.socket failed.
See "systemctl status virtlogd-admin.socket" and "journalctl -xe" for
details.
virtlogd-admin.socket couldn't start.
A dependency job for virtlogd.service failed. See 'journalctl -xe' for
details.
invoke-rc.d: initscript virtlogd, action "start" failed.
● virtlogd.service - Virtual machine log manager
   Loaded: loaded (/lib/systemd/system/virtlogd.service; indirect; vendor
preset: enabled)
   Active: active (running) since Thu 2018-08-09 09:35:36 UTC; 50s ago
     Docs: man:virtlogd(8)
           https://libvirt.org
 Main PID: 6774 (virtlogd)
   CGroup: /system.slice/virtlogd.service
           └─6774 /usr/sbin/virtlogd

Aug 09 09:35:38 debian-stretch systemd[1]: virtlogd.service: Failed to
reset devices.list: Operation not permitted
Aug 09 09:35:41 debian-stretch systemd[1]: virtlogd.service: Failed to
reset devices.list: Operation not permitted
Aug 09 09:36:21 debian-stretch systemd[1]: virtlogd.service: Failed to
reset devices.list: Operation not permitted
Aug 09 09:36:25 debian-stretch systemd[1]: virtlogd.service: Failed to
reset devices.list: Operation not permitted
Aug 09 09:36:25 debian-stretch systemd[1]: Reloading Virtual machine log
manager.
Aug 09 09:36:25 debian-stretch systemd[1]: Reloaded Virtual machine log
manager.
Aug 09 09:36:26 debian-stretch systemd[1]: virtlogd.service: Failed to
reset devices.list: Operation not permitted
Aug 09 09:36:26 debian-stretch systemd[1]: virtlogd.service: Failed to
reset devices.list: Operation not permitted
Aug 09 09:36:27 debian-stretch systemd[1]: Dependency failed for Virtual
machine log manager.
Aug 09 09:36:27 debian-stretch systemd[1]: virtlogd.service: Job
virtlogd.service/start failed with result 'dependency'.
dpkg: error processing package libvirt-daemon-system (--configure):
 subprocess installed post-installation script returned error exit status 1
Processing triggers for ca-certificates (20170717) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Processing triggers for systemd (232-25+deb9u4) ...
Errors were encountered while processing:
 libvirt-daemon-system
E: Sub-process /usr/bin/dpkg returned an error code (1)


Status after install:

# systemctl status virtlogd.service virtlogd.socket virtlogd-admin.socket
virtlockd.service virtlockd.socket virtlockd-admin.socket --no-pager
--lines 2
● virtlogd.service - Virtual machine log manager
   Loaded: loaded (/lib/systemd/system/virtlogd.service; indirect; vendor
preset: enabled)
   Active: active (running) since Thu 2018-08-09 09:35:36 UTC; 1min 11s ago
     Docs: man:virtlogd(8)
           https://libvirt.org
 Main PID: 6774 (virtlogd)
    Tasks: 2 (limit: 4915)
   CGroup: /system.slice/virtlogd.service
           └─6774 /usr/sbin/virtlogd

Aug 09 09:36:27 debian-stretch systemd[1]: virtlogd.service: Job
virtlogd.service/start failed with result 'dependency'.
Aug 09 09:36:31 debian-stretch systemd[1]: virtlogd.service: Failed to
reset devices.list: Operation not permitted

● virtlogd.socket - Virtual machine log manager socket
   Loaded: loaded (/lib/systemd/system/virtlogd.socket; enabled; vendor
preset: enabled)
   Active: active (running) since Thu 2018-08-09 09:35:36 UTC; 1min 11s ago
   Listen: /var/run/libvirt/virtlogd-sock (Stream)

Aug 09 09:35:36 debian-stretch systemd[1]: Listening on Virtual machine log
manager socket.

● virtlogd-admin.socket - Virtual machine log manager socket
   Loaded: loaded (/lib/systemd/system/virtlogd-admin.socket; enabled;
vendor preset: enabled)
   Active: inactive (dead)
   Listen: /var/run/libvirt/virtlogd-admin-sock (Stream)

Aug 09 09:36:27 debian-stretch systemd[1]: virtlogd-admin.socket: Socket
service virtlogd.service already active, refusing.
Aug 09 09:36:27 debian-stretch systemd[1]: Failed to listen on Virtual
machine log manager socket.

● virtlockd.service - Virtual machine lock manager
   Loaded: loaded (/lib/systemd/system/virtlockd.service; indirect; vendor
preset: enabled)
   Active: inactive (dead)
     Docs: man:virtlockd(8)
           https://libvirt.org

● virtlockd.socket - Virtual machine lock manager socket
   Loaded: loaded (/lib/systemd/system/virtlockd.socket; enabled; vendor
preset: enabled)
   Active: active (listening) since Thu 2018-08-09 09:36:26 UTC; 21s ago
   Listen: /var/run/libvirt/virtlockd-sock (Stream)

Aug 09 09:36:26 debian-stretch systemd[1]: Stopping Virtual machine lock
manager socket.
Aug 09 09:36:26 debian-stretch systemd[1]: Listening on Virtual machine
lock manager socket.

● virtlockd-admin.socket - Virtual machine lock manager admin socket
   Loaded: loaded (/lib/systemd/system/virtlockd-admin.socket; enabled;
vendor preset: enabled)
   Active: active (listening) since Thu 2018-08-09 09:36:26 UTC; 21s ago
   Listen: /var/run/libvirt/virtlockd-admin-sock (Stream)

Aug 09 09:36:26 debian-stretch systemd[1]: Listening on Virtual machine
lock manager admin socket.

[1]: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1786179

-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-libvirt-maintainers/attachments/20180809/e7a93252/attachment-0001.html>


More information about the Pkg-libvirt-maintainers mailing list