<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi,<br>
</p>
<p>I encountered the same problem.</p>
<p>It looks like libvirt denies write access to guest base image by
modifying the guest related apparmor profile once the guest runs
on a snapshot referring to this base image.</p>
<p>Here's what i tried:</p>
<pre>Guest runs on base image:
root@host/storage/vm# virsh domblklist test
Target Source
--------------------------------
vda /storage/vm/test.img
Generated aa profile not contains deny rule for base image:
root@host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
# DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
"/var/log/libvirt/**/test.log" w,
"/var/lib/libvirt/qemu/domain-test/monitor.sock" rw,
"/var/lib/libvirt/qemu/domain-8-test/*" rw,
"/var/run/libvirt/**/test.pid" rwk,
"/run/libvirt/**/test.pid" rwk,
"/var/run/libvirt/**/*.tunnelmigrate.dest.test" rw,
"/run/libvirt/**/*.tunnelmigrate.dest.test" rw,
"/storage/vm/test.img" rwk, <---------------------------------------
"/dev/vhost-net" rw,
"/var/lib/libvirt/qemu/domain-8-test/{,**}" rwk,
"/var/lib/libvirt/qemu/channel/target/domain-8-test/{,**}" rwk,
"/var/lib/libvirt/qemu/domain-8-test/master-key.aes" rwk,
"/dev/net/tun" rwk,
After generating the snapshot, there's still no deny rule (but we also not tried to merge back to base image yet):
root@host/storage/vm# virsh snapshot-create-as --domain test --name backup_overlay --no-metadata --atomic --disk-only --diskspec vda,snapshot=external
Domain snapshot backup_overlay created
root@host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
# DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
"/var/log/libvirt/**/test.log" w,
"/var/lib/libvirt/qemu/domain-test/monitor.sock" rw,
"/var/lib/libvirt/qemu/domain-8-test/*" rw,
"/var/run/libvirt/**/test.pid" rwk,
"/run/libvirt/**/test.pid" rwk,
"/var/run/libvirt/**/*.tunnelmigrate.dest.test" rw,
"/run/libvirt/**/*.tunnelmigrate.dest.test" rw,
"/storage/vm/test.img" rwk, <---------------------------------------
"/dev/pts/2" rw,
"/dev/pts/2" rw,
"/var/lib/libvirt/qemu/channel/target/domain-8-test/org.qemu.guest_agent.0" rw,
"/dev/vhost-net" rw,
"/storage/vm/test.backup_overlay" rwk,
When shutting down and starting guest again, host specific aa profile gets regenerated.
Now the deny rule is present. Note that guest runs on snapshot:
root@host/storage/vm# virsh domblklist test
Target Source
-------------------------------------------
vda /storage/vm/test.backup_overlay
root@host/storage/vm# virsh shutdown test
Domain test is being shutdown
root@host/storage/vm# virsh start test
Domain test started
root@host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
# DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
...
"/storage/vm/test.img" rk, <-------------------------------------------
# don't audit writes to readonly files
deny "/storage/vm/test.img" w, <----------------------------------------
"/dev/vhost-net" rw,
...
And it seems like this rule prevents us from merging snapshot back to base image:
root@host/storage/vm# virsh blockcommit test vda --active --pivoterror: internal error: unable to execute QEMU command 'block-commit': Could not reopen file: Permission denied
As soon as the guest aa profile gets disabled, blockcommit works as expected:
root@host/storage/vm# aa-disable /etc/apparmor.d/libvirt/libvirt-<uuid>
Disabling /etc/apparmor.d/libvirt/libvirt-<uuid>.
root@host/storage/vm# virsh blockcommit test vda --active --pivot
Successfully pivoted
After merge, stop and start guest, the deny rule is gone again:
root@host/storage/vm# virsh shutdown test
Domain test is being shutdown
root@host/storage/vm# virsh start test
Domain test started
root@host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
# DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
...
"/storage/vm/test.img" rwk,
"/dev/vhost-net" rw,
...
root@host/storage/vm# virsh domblklist test
Target Source
--------------------------------
vda /storage/vm/test.img
As stated above, the deny rule is not created immediately when creating the snapshot
if guest not gets stopped/started. But it gets added as soon as we try to merge back
to the base image:
root@host/storage/vm# virsh snapshot-create-as --domain test --name backup_overlay --no-metadata --atomic --disk-only --diskspec vda,snapshot=external
Domain snapshot backup_overlay created
root@host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
# DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
...
"/storage/vm/test.img" rwk,
...
root@host/storage/vm# virsh blockcommit test vda --active --pivoterror: internal error: unable to execute QEMU command 'block-commit': Could not reopen file: Permission denied
root@host/storage/vm# cat /etc/apparmor.d/libvirt/libvirt-<uuid>.files
# DO NOT EDIT THIS FILE DIRECTLY. IT IS MANAGED BY LIBVIRT.
...
"/storage/vm/test.img" rk,
# don't audit writes to readonly files
deny "/storage/vm/test.img" w, <--------------------------------------------
"/dev/pts/2" rw,
"/dev/pts/2" rw,
"/var/lib/libvirt/qemu/channel/target/domain-10-test/org.qemu.guest_agent.0" rw,
"/dev/vhost-net" rw,
"/storage/vm/test.img" rwk,
I fear this is not solvable by just fixing some apparmor base profiles because libvirt creates and modifies aa profiles on the fly via virt-aa-helper.
Would be good to know if this procedure fails as well on other distros if apparmor is enabled, then it's probably a general problem with libvirt and apparmor.
Thanks,
Kind Regards,
Robert
</pre>
</body>
</html>