[Pkg-libvirt-maintainers] Bug#981435: libvirt: stops on upgrade: internal error: Failed to load module 'libvirt_driver_qemu.so': libvirt.so.0: version `LIBVIRT_PRIVATE_6.9.0' not found (required by libvirt_driver_qemu.so)

[Andrea Bolognani]

On Sun, Jan 31, 2021 at 05:43:43PM +0800, Paul Wise wrote:
> Source: libvirt
> Version: 7.0.0-1
> Severity: serious
> 
> There is some sort of missing dependency that means that the daemon is
> stopped during certain kinds of upgrades. I use unattended-upgrades in
> the minimal-steps mode rather than upgrading all packages at once. As a
> result the order of upgrades seems to cause a failure to restart the
> libvirt systemd service. It looks like the libvirt-daemon-* packages
> need to be upgraded before the libvirt-daemon-system package.
> 
> I have included below the apt and systemd logs for this upgrade.
> 
> Log started: 2021-01-31  17:07:53
> apt-listchanges: Reading changelogs...
> apt-listchanges: Mailing root: apt-listchanges: changelogs for chianamo
> Preconfiguring packages ...
> apt-listchanges: Reading changelogs...
> Preconfiguring packages ...
> Preparing to unpack .../libvirt0-dbgsym_7.0.0-1_amd64.deb ...
> Unpacking libvirt0-dbgsym:amd64 (7.0.0-1) over (6.9.0-4) ...
> Preparing to unpack .../libvirt-daemon-system_7.0.0-1_amd64.deb ...
> Unpacking libvirt-daemon-system (7.0.0-1) over (6.9.0-4) ...
> Preparing to unpack .../libvirt-daemon_7.0.0-1_amd64.deb ...
> Unpacking libvirt-daemon (7.0.0-1) over (6.9.0-4) ...
> Preparing to unpack .../libvirt-clients_7.0.0-1_amd64.deb ...
> Unpacking libvirt-clients (7.0.0-1) over (6.9.0-4) ...
> Preparing to unpack .../libvirt0_7.0.0-1_amd64.deb ...
> Unpacking libvirt0:amd64 (7.0.0-1) over (6.9.0-4) ...
> Setting up libvirt0:amd64 (7.0.0-1) ...
> Setting up libvirt-daemon (7.0.0-1) ...
> Setting up libvirt-clients (7.0.0-1) ...
> Setting up libvirt0-dbgsym:amd64 (7.0.0-1) ...
> Setting up libvirt-daemon-system (7.0.0-1) ...
> Installing new version of config file /etc/apparmor.d/abstractions/libvirt-lxc ...
> Installing new version of config file /etc/apparmor.d/abstractions/libvirt-qemu ...
> Installing new version of config file /etc/default/libvirt-guests ...
> Installing new version of config file /etc/libvirt/qemu.conf ...
> Job for libvirtd.service failed because the control process exited with error code.
> See "systemctl status libvirtd.service" and "journalctl -xe" for details.
> virtlockd.service is a disabled or a static unit, not starting it.
> virtlogd.service is a disabled or a static unit, not starting it.
> Log ended: 2021-01-31  17:09:05
> 
> Log started: 2021-01-31  17:13:02
> apt-listchanges: Reading changelogs...
> apt-listchanges: Reading changelogs...
> Preparing to unpack .../libvirt-daemon-config-nwfilter_7.0.0-1_all.deb ...
> Unpacking libvirt-daemon-config-nwfilter (7.0.0-1) over (6.9.0-4) ...
> Setting up libvirt-daemon-config-nwfilter (7.0.0-1) ...
> Log ended: 2021-01-31  17:13:24
> 
> Log started: 2021-01-31  17:15:12
> apt-listchanges: Reading changelogs...
> apt-listchanges: Reading changelogs...
> Preparing to unpack .../libvirt-daemon-config-network_7.0.0-1_all.deb ...
> Unpacking libvirt-daemon-config-network (7.0.0-1) over (6.9.0-4) ...
> Setting up libvirt-daemon-config-network (7.0.0-1) ...
> Log ended: 2021-01-31  17:15:32
> 
> Log started: 2021-01-31  17:15:33
> apt-listchanges: Reading changelogs...
> apt-listchanges: Reading changelogs...
> Preparing to unpack .../libvirt-daemon-system-systemd_7.0.0-1_all.deb ...
> Unpacking libvirt-daemon-system-systemd (7.0.0-1) over (6.9.0-4) ...
> Setting up libvirt-daemon-system-systemd (7.0.0-1) ...
> Log ended: 2021-01-31  17:15:53
> 
> Log started: 2021-01-31  17:15:54
> apt-listchanges: Reading changelogs...
> apt-listchanges: Reading changelogs...
> Preparing to unpack .../libvirt-daemon-driver-qemu_7.0.0-1_amd64.deb ...
> Unpacking libvirt-daemon-driver-qemu (7.0.0-1) over (6.9.0-4) ...
> Setting up libvirt-daemon-driver-qemu (7.0.0-1) ...
> Processing triggers for man-db (2.9.3-2) ...
> Log ended: 2021-01-31  17:16:22
> 
> Jan 31 17:08:34 kernel: audit: type=1400 audit(1612084114.263:107): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="libvirtd" pid=1318234 comm="apparmor_parser"
> Jan 31 17:08:34 kernel: audit: type=1400 audit(1612084114.263:108): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="libvirtd//qemu_bridge_helper" pid=1318234 comm="apparmor_parser"
> Jan 31 17:08:34 audit[1318234]: AVC apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="libvirtd" pid=1318234 comm="apparmor_parser"
> Jan 31 17:08:34 audit[1318234]: AVC apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="libvirtd//qemu_bridge_helper" pid=1318234 comm="apparmor_parser"
> Jan 31 17:08:34 systemd[1]: Reloading.
> Jan 31 17:08:36 systemd[1]: Stopping Virtualization daemon...
> Jan 31 17:08:36 systemd[1]: libvirtd.service: Succeeded.
> Jan 31 17:08:36 systemd[1]: Stopped Virtualization daemon.
> Jan 31 17:08:36 systemd[1]: Starting Virtualization daemon...
> Jan 31 17:08:36 libvirtd[1318287]: libvirt version: 7.0.0, package: 1 (Andrea Bolognani <eof at kiyuko.org> Thu, 28 Jan 2021 22:06:43 +0100)
> Jan 31 17:08:36 libvirtd[1318287]: hostname: chianamo
> Jan 31 17:08:36 libvirtd[1318287]: internal error: Failed to load module '/usr/lib/x86_64-linux-gnu/libvirt/connection-driver/libvirt_driver_qemu.so': /usr/lib/x86_64-linux-gnu/libvirt.so.0: version `LIBVIRT_PRIVATE_6.9.0' not found (required by /usr/lib/x86_64-linux-gnu/libvirt/connection-driver/libvirt_driver_qemu.so)
> Jan 31 17:08:36 systemd[1]: libvirtd.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
> Jan 31 17:08:36 systemd[1]: libvirtd.service: Failed with result 'exit-code'.
> Jan 31 17:08:36 systemd[1]: Failed to start Virtualization daemon.

Thanks for your report, Paul!

As I've never used unattended-upgrades myself, I'm not familiar with
it. Is there any chance you could provide some quick tips on how to
set up a reproducer environment? Specifically how to set up the same
upgrade strategy you're using, and whether it's possible to manually
trigger an unattended-upgrades run? That would help a lot!

As for the issue itself, I think it's caused by some of the
dependencies not being strict enough: in particular we have

  Package: libvirt-daemon
  Depends: libvirt-daemon-driver-qemu

In your case, libvirt-daemon-driver-qemu 6.9.0-4 satisifies the
constraint for libvirt-daemon 7.0.0-1, and so the first upgrade step
only upgrades the latter and leaves the former for later: what I
believe we really want instead is

  Package: libvirt-daemon
  Depends: libvirt-daemon-driver-qemu (= ${binary:Version})

so that we can be sure that all libvirt packages are upgraded in
lockstep. We already have this for some inter-source-package
dependencies, we're just not entirely consistent about it.

I'll start working on a MR right away.

-- 
Andrea Bolognani <eof at kiyuko.org>
Resistance is futile, you will be garbage collected.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-libvirt-maintainers/attachments/20210131/8d7c6136/attachment.sig>