[pkg-lxc-devel] Bug#947863: lxc: apparmor denied mount with unprivileged lxc
Pierre-Elliott Bécue
peb at debian.org
Wed Jan 1 15:25:24 GMT 2020
Control: tags -1 +wontfix
Le mercredi 01 janvier 2020 à 02:05:37+0100, Johannes 'josch' Schauer a écrit :
> Package: lxc
> Version: 1:3.1.0+really3.0.4-2
> Severity: normal
>
> Hi,
>
> when booting into a system started with unprivileged lxc, I'm getting
> the following errors:
>
> [ 5.818300] audit: type=1400 audit(1577840118.455:15): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=388 comm="(md-udevd)" flags="rw, rslave"
> [ 5.842226] audit: type=1400 audit(1577840118.479:16): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=389 comm="(md-udevd)" flags="rw, rslave"
> [ 5.875326] audit: type=1400 audit(1577840118.511:17): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=390 comm="(md-udevd)" flags="rw, rslave"
> [ 5.894556] audit: type=1400 audit(1577840118.531:18): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=391 comm="(md-udevd)" flags="rw, rslave"
> [ 5.919489] audit: type=1400 audit(1577840118.555:19): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=392 comm="(md-udevd)" flags="rw, rslave"
> [ 6.368326] audit: type=1400 audit(1577840119.003:20): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=398 comm="(modprobe)" flags="rw, rslave"
> [ 6.390053] audit: type=1400 audit(1577840119.027:21): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=400 comm="(d-logind)" flags="rw, rslave"
> [ 6.400681] audit: type=1400 audit(1577840119.035:22): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=405 comm="(modprobe)" flags="rw, rslave"
> [ 6.406682] audit: type=1400 audit(1577840119.043:23): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=406 comm="(d-logind)" flags="rw, rslave"
> [ 6.416232] audit: type=1400 audit(1577840119.051:24): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxc-container-default-cgns" name="/" pid=409 comm="(modprobe)" flags="rw, rslave"
>
> These errors keep repeating. The only way to silence them I found so far
> is to disable apparmor. But maybe the default lxc apparmor profile could
> be adjusted to prevent these errors?
>
> To reproduce the problem, please see the attached shell script which you
> can run without superuser privileges if you have
> kernel.unprivileged_userns_clone set to 1. After running the script, the
> errors above can be seen in qemu.log.
>
> Thanks!
>
> cheers, josch
> #!/bin/sh
>
> # Copyright 2019 Johannes 'josch' Schauer <josch at debian.org>
> #
> # Permission is hereby granted, free of charge, to any person obtaining a copy
> # of this software and associated documentation files (the "Software"), to deal
> # in the Software without restriction, including without limitation the rights
> # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> # copies of the Software, and to permit persons to whom the Software is
> # furnished to do so, subject to the following conditions:
> #
> # The above copyright notice and this permission notice shall be included in all
> # copies or substantial portions of the Software.
>
> set -exu
>
> # Reasons to use qemu:
> # - can also be run with autopkgtest backends that do not support
> # - can be run without superuser privileges outside of autopkgtest
>
> if [ -z ${AUTOPKGTEST_TMP+x} ]; then
> # if AUTOPKGTEST_TMP is not set, then this script is probably not
> # executed under autopkgtest, choose unshare mode for mmdebstrap so
> # that this script can be run without superuser privileges
> MODE="unshare"
> else
> # since AUTOPKGTEST_TMP is set, we assume that this script is executed
> # under autopkgtest --> switch to the temporary directory
> cd "$AUTOPKGTEST_TMP"
> # We have to use root mode on salsa ci because:
> # - unshare mode fails because /sys is mounted read-only
> # and kernel.unprivileged_userns_clone is not set to 1
> # - fakechroot mode fails because of #944929
> # - proot mode produces wrong permissions
> MODE="root"
> fi
>
> # setting up /etc/fstab, /etc/hostname and /etc/hosts is required to raise
> # network interfaces
> if [ ! -e debian-unstable-host.tar ]; then
> mmdebstrap --mode=$MODE --variant=apt \
> --include=openssh-server,systemd-sysv,ifupdown,netbase,isc-dhcp-client,udev,policykit-1,linux-image-amd64,lxc,uidmap,apparmor,bridge-utils,procps,iptables,iptables-persistent,psmisc \
> --customize-hook='echo host > "$1/etc/hostname"' \
> --customize-hook='echo "127.0.0.1 localhost host" > "$1/etc/hosts"' \
> --customize-hook='echo "/dev/vda1 / auto errors=remount-ro 0 1" > "$1/etc/fstab"' \
> --customize-hook='cat /etc/resolv.conf > "$1/etc/resolv.conf"' \
> unstable debian-unstable-host.tar
> fi
>
> # we prepare a second tarball now instead of later inside qemu because
> # running mmdebstrap without kvm just wastes cpu cycles
> if [ ! -e debian-unstable-container.tar ]; then
> mmdebstrap --mode=$MODE --variant=apt --include=dbus,udev,systemd-sysv,policykit-1 \
> --customize-hook='echo container > "$1/etc/hostname"' \
> --customize-hook='cat /etc/resolv.conf > "$1/etc/resolv.conf"' \
> unstable - \
> | python3 -c '
> import tarfile
> import sys
>
> with tarfile.open(fileobj=sys.stdin.buffer, mode="r|*") as in_tar, \
> tarfile.open(fileobj=sys.stdout.buffer, mode="w|") as out_tar:
> for member in in_tar:
> member.uid += 1000000
> member.gid += 1000000
> if member.isfile():
> with in_tar.extractfile(member) as file:
> out_tar.addfile(member, file)
> else:
> out_tar.addfile(member)
> ' > debian-unstable-container.tar
> fi
>
>
> echo 'net.ipv4.ip_forward=1' > sysctl.conf
>
> cat << END > rules.v4
> *nat
> :PREROUTING ACCEPT [0:0]
> :INPUT ACCEPT [0:0]
> :OUTPUT ACCEPT [0:0]
> :POSTROUTING ACCEPT [0:0]
> -A PREROUTING -i eth0 -p tcp -m tcp --dport 8002 -j DNAT --to-destination 10.0.0.2:8003
> -A POSTROUTING -o eth0 -j MASQUERADE
> COMMIT
> *filter
> :INPUT ACCEPT [0:0]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [0:0]
> COMMIT
> END
>
> # extlinux config to boot from /dev/vda1 with predictable network interface
> # naming and a serial console for logging
> cat << END > extlinux.conf
> default linux
> timeout 0
>
> label linux
> kernel /vmlinuz
> append initrd=/initrd.img root=/dev/vda1 net.ifnames=0 console=ttyS0
> END
>
> # network interface config
> # we can use eth0 because we boot with net.ifnames=0 for predictable interface
> # names
> cat << END > interfaces
> auto lo
> iface lo inet loopback
>
> auto eth0
> iface eth0 inet dhcp
>
> auto lxcbridge
> iface lxcbridge inet static
> address 10.0.0.1
> netmask 255.255.255.0
> bridge_stp off
> bridge_waitport 0
> bridge_fd 0
> bridge_ports none
> END
>
> echo 'root:1000000:65536' >> subuid
> echo 'root:1000000:65536' >> subgid
>
> cat << END > config
> lxc.include = /usr/share/lxc/config/common.conf
> lxc.include = /usr/share/lxc/config/userns.conf
> lxc.uts.name = container
> lxc.rootfs.path = dir:/srv/container
> lxc.idmap = u 0 1000000 65536
> lxc.idmap = g 0 1000000 65536
> lxc.net.0.ipv4.address.address = 10.0.0.2/24
> lxc.net.0.hwaddr = ee:ec:fa:e9:56:7d
> lxc.net.0.type = veth
> lxc.net.0.flags = up
> lxc.net.0.link = lxcbridge
> lxc.net.0.name = eth0
> lxc.net.0.mtu = 1500
> lxc.net.0.ipv4.gateway = 10.0.0.1
> lxc.start.auto = 1
> lxc.tty.max = 1
> END
>
> if [ ! -e id_rsa ]; then
> ssh-keygen -q -t rsa -f ./id_rsa -N ""
> fi
>
> # use guestfish to prepare the host system
> #
> # - create a single 2G partition and unpack the rootfs tarball into it
> # - copy the public key into .ssh/authorized_keys for the root user
> # - copy in extlinux.conf and /etc/network/interfaces
> # - unpack the tarball of the container into /srv/container
> # - put a syslinux MBR into the first 440 bytes of the drive
> # - install extlinux and make partition bootable
> #
> # useful stuff to debug any errors:
> # LIBGUESTFS_BACKEND_SETTINGS=force_tcg
> # libguestfs-test-tool || true
> # export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
> guestfish -N host.img=disk:2G -- \
> part-disk /dev/sda mbr : \
> mkfs ext2 /dev/sda1 : \
> mount /dev/sda1 / : \
> tar-in debian-unstable-host.tar / : \
> mkdir /root/.ssh : \
> upload id_rsa.pub /root/.ssh/authorized_keys : \
> chown 0 0 /root/.ssh/authorized_keys : \
> copy-in extlinux.conf / : \
> copy-in interfaces /etc/network : \
> copy-in sysctl.conf /etc : \
> copy-in rules.v4 /etc/iptables : \
> copy-in subuid /etc : \
> copy-in subgid /etc : \
> mkdir /var/lib/lxc/container : \
> copy-in config /var/lib/lxc/container : \
> mkdir /srv/container : \
> tar-in debian-unstable-container.tar /srv/container : \
> upload /usr/lib/SYSLINUX/mbr.bin /mbr.bin : \
> copy-file-to-device /mbr.bin /dev/sda size:440 : \
> rm /mbr.bin : \
> extlinux / : \
> sync : \
> umount / : \
> part-set-bootable /dev/sda 1 true : \
> shutdown
>
>
> # start the host system
> # redirect tcp connections on port 10011 localhost to the host system port 22
> # redirect all output to a file
> # run in the background
> qemu-system-x86_64 \
> -enable-kvm \
> -no-user-config \
> -object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 \
> -m 1G \
> -net nic,model=virtio \
> -nographic \
> -serial mon:stdio \
> -net user,hostfwd=tcp:127.0.0.1:10022-:22,hostfwd=tcp:127.0.0.1:8001-:8002 \
> -drive file=host.img,format=raw,if=virtio \
> >qemu.log </dev/null 2>&1 &
>
> # store the pid
> QEMUPID=$!
>
> # show the log and kill qemu in case the script exits first
> trap "cat --show-nonprinting qemu.log; kill $QEMUPID" EXIT
>
> # the default ssh command does not store known hosts and even ignores host keys
> # it identifies itself with the rsa key generated above
> # pseudo terminal allocation is disabled or otherwise, programs executed via
> # ssh might wait for input on stdin of the ssh process
> ssh="ssh -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no -i id_rsa -T"
>
> # we use sleepenh to make sure that we wait the right number of seconds
> # independent on how long the command took beforehand
> TIMESTAMP=$(sleepenh 0 || [ $? -eq 1 ])
> # the timeout in seconds
> TIMEOUT=5
> # the maximum number of tries
> NUM_TRIES=20
> i=0
> while true; do
> rv=0
> $ssh -p 10022 -o ConnectTimeout=$TIMEOUT root at localhost echo success || rv=1
> # with an exit code of zero, the ssh connection was successful
> # and we break out of the loop
> [ $rv -eq 0 ] && break
> # if the command before took less than $TIMEOUT seconds, wait the remaining time
> TIMESTAMP=$(sleepenh $TIMESTAMP $TIMEOUT || [ $? -eq 1 ]);
> # increment the counter and break out of the loop if we tried
> # too often
> i=$((i+1))
> if [ $i -ge $NUM_TRIES ]; then
> break
> fi
> done
>
> # if all tries were exhausted, the process failed
> if [ $i -eq $NUM_TRIES ]; then
> echo "timeout reached: unable to connect to qemu via ssh"
> exit 1
> fi
>
> # ip of host system outside qemu: 10.0.2.2
> # ip of host system inside qemu: 10.0.2.15
> # ip of container: 10.0.0.2
>
> # execute a shell script on the host system
> cat << END > expected
> NAME STATE AUTOSTART IPV4 UNPRIVILEGED
> container RUNNING 1 10.0.0.2 true
> END
>
> $ssh -p 10022 root at localhost lxc-ls --fancy --fancy-format=NAME,STATE,AUTOSTART,IPV4,UNPRIVILEGED | diff -u expected -
>
> $ssh -p 10022 root at localhost lxc-attach --name=container -- apt-get update
> $ssh -p 10022 root at localhost lxc-attach --name=container -- apt-get install --no-install-recommends --yes netcat-traditional procps
>
> $ssh -p 10022 root at localhost lxc-attach --name=container -- nc -lp 8003 | grep foobar &
> NCPID=$!
>
> sleep 5
>
> echo foobar | nc -q0 127.0.0.1 8001
>
> wait $NCPID
>
> LXCPID=$($ssh -p 10022 root at localhost lxc-info --name container | awk '/^PID:/ { print $2; }')
> $ssh -p 10022 root at localhost ps -jf -u 1000000 | awk "\$2 == $LXCPID" | grep /sbin/init
> $ssh -p 10022 root at localhost lxc-attach --name=container -- ps -jf -u 0 | grep /sbin/init
>
> trap - EXIT
>
> $ssh -p 10022 root at localhost systemctl poweroff || true
>
> wait $QEMUPID
>
> # debian-unstable-container.tar debian-unstable-host.tar id_rsa id_rsa.pub qemu.log
> for f in extlinux.conf interfaces host.img; do
> rm "$f"
> done
Dear Josch,
I'm sorry but lxc unprivileged containers can't run with any apparmor
profile. You have to set this parameter to unconfined for your
unprivileged containers. Setting a default profile for unconfined
containers is a hard thing as only etc/default/lxc.conf is an option,
but it'd also apply to privileged containers.
With best regards,
--
Pierre-Elliott Bécue
GPG: 9AE0 4D98 6400 E3B6 7528 F493 0D44 2664 1949 74E2
It's far easier to fight for one's principles than to live up to them.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-lxc-devel/attachments/20200101/111758d3/attachment.sig>
More information about the Pkg-lxc-devel
mailing list