[Pkg-libvirt-maintainers] Bug#932456: (no subject)

[Robert Niederreiter]

Hi,

I encountered the same problem.

It looks like libvirt denies write access to guest base image by 
modifying the guest related apparmor profile once the guest runs on a 
snapshot referring to this base image.

Here's what i tried:

Guest runs on base image:

     root at host/storage/vm# virsh domblklist test
      Target   Source
     --------------------------------
      vda      /storage/vm/test.img

Generated aa profile not contains deny rule for base image:

     root at host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
     # DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
       "/var/log/libvirt/**/test.log" w,
       "/var/lib/libvirt/qemu/domain-test/monitor.sock" rw,
       "/var/lib/libvirt/qemu/domain-8-test/*" rw,
       "/var/run/libvirt/**/test.pid" rwk,
       "/run/libvirt/**/test.pid" rwk,
       "/var/run/libvirt/**/*.tunnelmigrate.dest.test" rw,
       "/run/libvirt/**/*.tunnelmigrate.dest.test" rw,
       "/storage/vm/test.img" rwk,  <---------------------------------------
       "/dev/vhost-net" rw,
       "/var/lib/libvirt/qemu/domain-8-test/{,**}" rwk,
       "/var/lib/libvirt/qemu/channel/target/domain-8-test/{,**}" rwk,
       "/var/lib/libvirt/qemu/domain-8-test/master-key.aes" rwk,
       "/dev/net/tun" rwk,

After generating the snapshot, there's still no deny rule (but we also not tried to merge back to base image yet):

     root at host/storage/vm# virsh snapshot-create-as --domain test --name backup_overlay --no-metadata --atomic --disk-only --diskspec vda,snapshot=external
     Domain snapshot backup_overlay created

     root at host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
     # DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
       "/var/log/libvirt/**/test.log" w,
       "/var/lib/libvirt/qemu/domain-test/monitor.sock" rw,
       "/var/lib/libvirt/qemu/domain-8-test/*" rw,
       "/var/run/libvirt/**/test.pid" rwk,
       "/run/libvirt/**/test.pid" rwk,
       "/var/run/libvirt/**/*.tunnelmigrate.dest.test" rw,
       "/run/libvirt/**/*.tunnelmigrate.dest.test" rw,
       "/storage/vm/test.img" rwk,  <---------------------------------------
       "/dev/pts/2" rw,
       "/dev/pts/2" rw,
       "/var/lib/libvirt/qemu/channel/target/domain-8-test/org.qemu.guest_agent.0" rw,
       "/dev/vhost-net" rw,
       "/storage/vm/test.backup_overlay" rwk,

When shutting down and starting guest again, host specific aa profile gets regenerated.
Now the deny rule is present. Note that guest runs on snapshot:

     root at host/storage/vm# virsh domblklist test
      Target   Source
     -------------------------------------------
      vda      /storage/vm/test.backup_overlay

     root at host/storage/vm# virsh shutdown test
     Domain test is being shutdown

     root at host/storage/vm# virsh start test
     Domain test started

     root at host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
     # DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
       ...
       "/storage/vm/test.img" rk,   <-------------------------------------------
       # don't audit writes to readonly files
       deny "/storage/vm/test.img" w,  <----------------------------------------
       "/dev/vhost-net" rw,
       ...

And it seems like this rule prevents us from merging snapshot back to base image:

     root at host/storage/vm# virsh blockcommit test vda --active --pivoterror: internal error: unable to execute QEMU command 'block-commit': Could not reopen file: Permission denied

As soon as the guest aa profile gets disabled, blockcommit works as expected:

     root at host/storage/vm# aa-disable /etc/apparmor.d/libvirt/libvirt-<uuid>
     Disabling /etc/apparmor.d/libvirt/libvirt-<uuid>.

     root at host/storage/vm# virsh blockcommit test vda --active --pivot
     Successfully pivoted

After merge, stop and start guest, the deny rule is gone again:

     root at host/storage/vm# virsh shutdown test
     Domain test is being shutdown

     root at host/storage/vm# virsh start test
     Domain test started

     root at host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
     # DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
       ...
       "/storage/vm/test.img" rwk,
       "/dev/vhost-net" rw,
       ...

     root at host/storage/vm# virsh domblklist test
      Target   Source
     --------------------------------
      vda      /storage/vm/test.img

As stated above, the deny rule is not created immediately when creating the snapshot
if guest not gets stopped/started. But it gets added as soon as we try to merge back
to the base image:

     root at host/storage/vm# virsh snapshot-create-as --domain test --name backup_overlay --no-metadata --atomic --disk-only --diskspec vda,snapshot=external
     Domain snapshot backup_overlay created

     root at host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
     # DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
       ...
       "/storage/vm/test.img" rwk,
       ...

     root at host/storage/vm# virsh blockcommit test vda --active --pivoterror: internal error: unable to execute QEMU command 'block-commit': Could not reopen file: Permission denied

     root at host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
     # DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
       ...
       "/storage/vm/test.img" rk,
       # don't audit writes to readonly files
       deny "/storage/vm/test.img" w,  <--------------------------------------------
       "/dev/pts/2" rw,
       "/dev/pts/2" rw,
       "/var/lib/libvirt/qemu/channel/target/domain-10-test/org.qemu.guest_agent.0" rw,
       "/dev/vhost-net" rw,
       "/storage/vm/test.img" rwk,

I fear this is not solvable by just fixing some apparmor base profiles because libvirt creates and modifies aa profiles on the fly via virt-aa-helper.
Would be good to know if this procedure fails as well on other distros if apparmor is enabled, then it's probably a general problem with libvirt and apparmor.

Thanks,

Kind Regards,

Robert

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/pkg-libvirt-maintainers/attachments/20191013/b0ed1447/attachment.html>