[pkg-lxc-devel] Bug#947863: lxc: apparmor denied mount with unprivileged lxc

Pierre-Elliott Bécue peb at debian.org
Wed Jan 1 15:25:24 GMT 2020


Control: tags -1 +wontfix

Le mercredi 01 janvier 2020 à 02:05:37+0100, Johannes 'josch' Schauer a écrit :
> Package: lxc
> Version: 1:3.1.0+really3.0.4-2
> Severity: normal
> 
> Hi,
> 
> when booting into a system started with unprivileged lxc, I'm getting
> the following errors:
> 
> [    5.818300] audit: type=1400 audit(1577840118.455:15): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=388 comm="(md-udevd)" flags="rw, rslave"
> [    5.842226] audit: type=1400 audit(1577840118.479:16): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=389 comm="(md-udevd)" flags="rw, rslave"
> [    5.875326] audit: type=1400 audit(1577840118.511:17): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=390 comm="(md-udevd)" flags="rw, rslave"
> [    5.894556] audit: type=1400 audit(1577840118.531:18): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=391 comm="(md-udevd)" flags="rw, rslave"
> [    5.919489] audit: type=1400 audit(1577840118.555:19): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=392 comm="(md-udevd)" flags="rw, rslave"
> [    6.368326] audit: type=1400 audit(1577840119.003:20): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=398 comm="(modprobe)" flags="rw, rslave"
> [    6.390053] audit: type=1400 audit(1577840119.027:21): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=400 comm="(d-logind)" flags="rw, rslave"
> [    6.400681] audit: type=1400 audit(1577840119.035:22): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=405 comm="(modprobe)" flags="rw, rslave"
> [    6.406682] audit: type=1400 audit(1577840119.043:23): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=406 comm="(d-logind)" flags="rw, rslave"
> [    6.416232] audit: type=1400 audit(1577840119.051:24): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=409 comm="(modprobe)" flags="rw, rslave"
> 
> These errors keep repeating. The only way to silence them I found so far
> is to disable apparmor. But maybe the default lxc apparmor profile could
> be adjusted to prevent these errors?
> 
> To reproduce the problem, please see the attached shell script which you
> can run without superuser privileges if you have
> kernel.unprivileged_userns_clone set to 1. After running the script, the
> errors above can be seen in qemu.log.
> 
> Thanks!
> 
> cheers, josch

> #!/bin/sh
> 
> # Copyright 2019 Johannes 'josch' Schauer <josch at debian.org>
> #
> # Permission is hereby granted, free of charge, to any person obtaining a copy
> # of this software and associated documentation files (the "Software"), to deal
> # in the Software without restriction, including without limitation the rights
> # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> # copies of the Software, and to permit persons to whom the Software is
> # furnished to do so, subject to the following conditions:
> #
> # The above copyright notice and this permission notice shall be included in all
> # copies or substantial portions of the Software.
> 
> set -exu
> 
> # Reasons to use qemu:
> #   - can also be run with autopkgtest backends that do not support 
> #   - can be run without superuser privileges outside of autopkgtest
> 
> if [ -z ${AUTOPKGTEST_TMP+x} ]; then
> 	# if AUTOPKGTEST_TMP is not set, then this script is probably not
> 	# executed under autopkgtest, choose unshare mode for mmdebstrap so
> 	# that this script can be run without superuser privileges
> 	MODE="unshare"
> else
> 	# since AUTOPKGTEST_TMP is set, we assume that this script is executed
> 	# under autopkgtest --> switch to the temporary directory
> 	cd "$AUTOPKGTEST_TMP"
> 	# We have to use root mode on salsa ci because:
> 	#  - unshare mode fails because /sys is mounted read-only
> 	#    and kernel.unprivileged_userns_clone is not set to 1
> 	#  - fakechroot mode fails because of #944929
> 	#  - proot mode produces wrong permissions
> 	MODE="root"
> fi
> 
> # setting up /etc/fstab, /etc/hostname and /etc/hosts is required to raise
> # network interfaces
> if [ ! -e debian-unstable-host.tar ]; then
> mmdebstrap --mode=$MODE --variant=apt \
> 	--include=openssh-server,systemd-sysv,ifupdown,netbase,isc-dhcp-client,udev,policykit-1,linux-image-amd64,lxc,uidmap,apparmor,bridge-utils,procps,iptables,iptables-persistent,psmisc \
> 	--customize-hook='echo host > "$1/etc/hostname"' \
> 	--customize-hook='echo "127.0.0.1 localhost host" > "$1/etc/hosts"' \
> 	--customize-hook='echo "/dev/vda1 / auto errors=remount-ro 0 1" > "$1/etc/fstab"' \
> 	--customize-hook='cat /etc/resolv.conf > "$1/etc/resolv.conf"' \
> 	unstable debian-unstable-host.tar
> fi
> 
> # we prepare a second tarball now instead of later inside qemu because
> # running mmdebstrap without kvm just wastes cpu cycles
> if [ ! -e debian-unstable-container.tar ]; then
> mmdebstrap --mode=$MODE --variant=apt --include=dbus,udev,systemd-sysv,policykit-1 \
> 	--customize-hook='echo container > "$1/etc/hostname"' \
> 	--customize-hook='cat /etc/resolv.conf > "$1/etc/resolv.conf"' \
> 	unstable - \
> 	| python3 -c '
> import tarfile
> import sys
> 
> with tarfile.open(fileobj=sys.stdin.buffer, mode="r|*") as in_tar, \
>     tarfile.open(fileobj=sys.stdout.buffer, mode="w|") as out_tar:
>     for member in in_tar:
>         member.uid += 1000000
>         member.gid += 1000000
>         if member.isfile():
>             with in_tar.extractfile(member) as file:
>                 out_tar.addfile(member, file)
>         else:
>             out_tar.addfile(member)
> ' > debian-unstable-container.tar
> fi
> 
> 
> echo 'net.ipv4.ip_forward=1' > sysctl.conf
> 
> cat << END > rules.v4
> *nat
> :PREROUTING ACCEPT [0:0]
> :INPUT ACCEPT [0:0]
> :OUTPUT ACCEPT [0:0]
> :POSTROUTING ACCEPT [0:0]
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 8002 -j DNAT --to-destination 10.0.0.2:8003
> -A POSTROUTING -o eth0 -j MASQUERADE
> COMMIT
> *filter
> :INPUT ACCEPT [0:0]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [0:0]
> COMMIT
> END
> 
> # extlinux config to boot from /dev/vda1 with predictable network interface
> # naming and a serial console for logging
> cat << END > extlinux.conf
> default linux
> timeout 0
> 
> label linux
> kernel /vmlinuz
> append initrd=/initrd.img root=/dev/vda1 net.ifnames=0 console=ttyS0
> END
> 
> # network interface config
> # we can use eth0 because we boot with net.ifnames=0 for predictable interface
> # names
> cat << END > interfaces
> auto lo
> iface lo inet loopback
> 
> auto eth0
> iface eth0 inet dhcp
> 
> auto lxcbridge
> iface lxcbridge inet static
>   address 10.0.0.1
>   netmask 255.255.255.0
>   bridge_stp off
>   bridge_waitport 0
>   bridge_fd 0
>   bridge_ports none
> END
> 
> echo 'root:1000000:65536' >> subuid
> echo 'root:1000000:65536' >> subgid
> 
> cat << END > config
> lxc.include = /usr/share/lxc/config/common.conf
> lxc.include = /usr/share/lxc/config/userns.conf
> lxc.uts.name = container
> lxc.rootfs.path = dir:/srv/container
> lxc.idmap = u 0 1000000 65536
> lxc.idmap = g 0 1000000 65536
> lxc.net.0.ipv4.address.address = 10.0.0.2/24
> lxc.net.0.hwaddr = ee:ec:fa:e9:56:7d
> lxc.net.0.type = veth
> lxc.net.0.flags = up
> lxc.net.0.link = lxcbridge
> lxc.net.0.name = eth0
> lxc.net.0.mtu = 1500
> lxc.net.0.ipv4.gateway = 10.0.0.1
> lxc.start.auto = 1
> lxc.tty.max = 1
> END
> 
> if [ ! -e id_rsa ]; then
> ssh-keygen -q -t rsa -f ./id_rsa -N ""
> fi
> 
> # use guestfish to prepare the host system
> #
> #  - create a single 2G partition and unpack the rootfs tarball into it
> #  - copy the public key into .ssh/authorized_keys for the root user
> #  - copy in extlinux.conf and /etc/network/interfaces
> #  - unpack the tarball of the container into /srv/container
> #  - put a syslinux MBR into the first 440 bytes of the drive
> #  - install extlinux and make partition bootable
> #
> # useful stuff to debug any errors:
> #   LIBGUESTFS_BACKEND_SETTINGS=force_tcg
> #   libguestfs-test-tool || true
> #   export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
> guestfish -N host.img=disk:2G -- \
> 	part-disk /dev/sda mbr : \
> 	mkfs ext2 /dev/sda1 : \
> 	mount /dev/sda1 / : \
> 	tar-in debian-unstable-host.tar / : \
> 	mkdir /root/.ssh : \
> 	upload id_rsa.pub /root/.ssh/authorized_keys : \
> 	chown 0 0 /root/.ssh/authorized_keys : \
> 	copy-in extlinux.conf / : \
> 	copy-in interfaces /etc/network : \
> 	copy-in sysctl.conf /etc : \
> 	copy-in rules.v4 /etc/iptables : \
> 	copy-in subuid /etc : \
> 	copy-in subgid /etc : \
> 	mkdir /var/lib/lxc/container : \
> 	copy-in config /var/lib/lxc/container : \
> 	mkdir /srv/container : \
> 	tar-in debian-unstable-container.tar /srv/container : \
> 	upload /usr/lib/SYSLINUX/mbr.bin /mbr.bin : \
> 	copy-file-to-device /mbr.bin /dev/sda size:440 : \
> 	rm /mbr.bin : \
> 	extlinux / : \
> 	sync : \
> 	umount / : \
> 	part-set-bootable /dev/sda 1 true : \
> 	shutdown
> 
> 
> # start the host system
> # redirect tcp connections on port 10011 localhost to the host system port 22
> # redirect all output to a file
> # run in the background
> qemu-system-x86_64 \
> 	-enable-kvm \
> 	-no-user-config \
> 	-object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 \
> 	-m 1G \
> 	-net nic,model=virtio \
> 	-nographic \
> 	-serial mon:stdio \
> 	-net user,hostfwd=tcp:127.0.0.1:10022-:22,hostfwd=tcp:127.0.0.1:8001-:8002 \
> 	-drive file=host.img,format=raw,if=virtio \
> 	>qemu.log </dev/null 2>&1 &
> 
> # store the pid
> QEMUPID=$!
> 
> # show the log and kill qemu in case the script exits first
> trap "cat --show-nonprinting qemu.log; kill $QEMUPID" EXIT
> 
> # the default ssh command does not store known hosts and even ignores host keys
> # it identifies itself with the rsa key generated above
> # pseudo terminal allocation is disabled or otherwise, programs executed via
> # ssh might wait for input on stdin of the ssh process
> ssh="ssh -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no -i id_rsa -T"
> 
> # we use sleepenh to make sure that we wait the right number of seconds
> # independent on how long the command took beforehand
> TIMESTAMP=$(sleepenh 0 || [ $? -eq 1 ])
> # the timeout in seconds
> TIMEOUT=5
> # the maximum number of tries
> NUM_TRIES=20
> i=0
> while true; do
> 	rv=0
> 	$ssh -p 10022 -o ConnectTimeout=$TIMEOUT root at localhost echo success || rv=1
> 	# with an exit code of zero, the ssh connection was successful
> 	# and we break out of the loop
> 	[ $rv -eq 0 ] && break
> 	# if the command before took less than $TIMEOUT seconds, wait the remaining time
> 	TIMESTAMP=$(sleepenh $TIMESTAMP $TIMEOUT || [ $? -eq 1 ]);
> 	# increment the counter and break out of the loop if we tried
> 	# too often
> 	i=$((i+1))
> 	if [ $i -ge $NUM_TRIES ]; then
> 		break
> 	fi
> done
> 
> # if all tries were exhausted, the process failed
> if [ $i -eq $NUM_TRIES ]; then
> 	echo "timeout reached: unable to connect to qemu via ssh"
> 	exit 1
> fi
> 
> # ip of host system outside qemu: 10.0.2.2
> # ip of host system inside qemu: 10.0.2.15
> # ip of container: 10.0.0.2
> 
> # execute a shell script on the host system
> cat << END > expected
> NAME      STATE   AUTOSTART IPV4     UNPRIVILEGED 
> container RUNNING 1         10.0.0.2 true         
> END
> 
> $ssh -p 10022 root at localhost lxc-ls --fancy --fancy-format=NAME,STATE,AUTOSTART,IPV4,UNPRIVILEGED | diff -u expected -
> 
> $ssh -p 10022 root at localhost lxc-attach --name=container -- apt-get update
> $ssh -p 10022 root at localhost lxc-attach --name=container -- apt-get install --no-install-recommends --yes netcat-traditional procps
> 
> $ssh -p 10022 root at localhost lxc-attach --name=container -- nc -lp 8003 | grep foobar &
> NCPID=$!
> 
> sleep 5
> 
> echo foobar | nc -q0 127.0.0.1 8001
> 
> wait $NCPID
> 
> LXCPID=$($ssh -p 10022 root at localhost lxc-info --name container | awk '/^PID:/ { print $2; }')
> $ssh -p 10022 root at localhost ps -jf -u 1000000 | awk "\$2 == $LXCPID" | grep /sbin/init
> $ssh -p 10022 root at localhost lxc-attach --name=container -- ps -jf -u 0 | grep /sbin/init
> 
> trap - EXIT
> 
> $ssh -p 10022 root at localhost systemctl poweroff || true
> 
> wait $QEMUPID
> 
> # debian-unstable-container.tar debian-unstable-host.tar id_rsa id_rsa.pub qemu.log
> for f in extlinux.conf interfaces host.img; do
> 	rm "$f"
> done

Dear Josch,

I'm sorry but lxc unprivileged containers can't run with any apparmor
profile. You have to set this parameter to unconfined for your
unprivileged containers. Setting a default profile for unconfined
containers is a hard thing as only etc/default/lxc.conf is an option,
but it'd also apply to privileged containers.

With best regards,

-- 
Pierre-Elliott Bécue
GPG: 9AE0 4D98 6400 E3B6 7528  F493 0D44 2664 1949 74E2
It's far easier to fight for one's principles than to live up to them.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-lxc-devel/attachments/20200101/111758d3/attachment.sig>


More information about the Pkg-lxc-devel mailing list