[pkg-bacula-devel] Running autopkgtest at home

[Sven Hartge]

Hi!

Here is my quick introduction on how to get autopkgtest running locally
without the need to create a complete CI environment:

I found that using lxc is the fastest and most lightweight method while
still allowing to run jobs with "breaks-testbed" or "container-isolation"
requirements.

qemu is harder to setup, may not run everywhere (needs KVM to be
reasonably fast) and the images take up more space. (Plus I never got it
running in the first place.)

schroot is the lightest of all, because no additional special network
setup is needed, but the autopkgtest-virt-chroot runner does not provide
any revert or isolation capabilities, making it unfit to run jobs
requiring them.

There is a Docker-based autopkgtest-runner in development, but so far no
progress has been made to merge this into the main branch.

To run LXC-based autopkgtests one needs a Linux-based system, obviously.

a) Installation:

 apt install autopkgtest lxc sudo

To make things easier later your user should be able to run the lxc-* and
autopkgtest-build-lxc commands as user root.

b) Configuration:

LXC needs an advanced network setup, so the running containers can have
Internet access.

Change /etc/lxc/default.conf to look like this:

lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:xx:xx:x

Then create the file /etc/default/lxc-net with the following content,
adapting the IPs to your liking:

USE_LXC_BRIDGE="true"
LXC_ADDR="172.31.255.1"
LXC_NETMASK="255.255.255.0"
LXC_NETWORK="172.31.255.0/24"
LXC_DHCP_RANGE="172.31.255.2,172.31.255.254"
LXC_DHCP_MAX="253"
LXC_DHCP_CONFILE=""
LXC_DOMAIN=""

LXC_IPV6_ADDR=""
LXC_IPV6_MASK=""
LXC_IPV6_NETWORK=""
LXC_IPV6_NAT="false"

Now we need to activate our setup. At first, restart apparmor to properly
import the setting LXC added:

 service apparmor restart

Next we restart lxc-net to get the network setup running:

 service lxc-net restart

There should be a lxcbr0 device available now:

5: lxcbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 00:16:3e:00:00:00 brd ff:ff:ff:ff:ff:ff
    inet 172.31.255.1/24 scope global lxcbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fe00:0/64 scope link 
       valid_lft forever preferred_lft forever

And a dnsmasq instance should be running on that interface:

dnsmasq  15049  0.0  0.0  26980   596 ?        S    Nov21   0:02 dnsmasq -u dnsmasq --strict-order --bind-interfaces --pid-file=/run/lxc/dnsmasq.pid --listen-address 172.31.255.1 --dhcp-range 172.31.255.2,172.31.255.254 --dhcp-lease-max=253 --dhcp-no-override --except-interface=lo --interface=lxcbr0 --dhcp-leasefile=/var/lib/misc/dnsmasq.lxcbr0.leases --dhcp-authoritative

It should also have create some iptables rules:

Filter:
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  lxcbr0 any     anywhere             anywhere             tcp dpt:domain
  366 24912 ACCEPT     udp  --  lxcbr0 any     anywhere             anywhere             udp dpt:domain
    0     0 ACCEPT     tcp  --  lxcbr0 any     anywhere             anywhere             tcp dpt:bootps
   33 10824 ACCEPT     udp  --  lxcbr0 any     anywhere             anywhere             udp dpt:bootps

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
34701  373M ACCEPT     all  --  any    lxcbr0  anywhere             anywhere            
22798 1259K ACCEPT     all  --  lxcbr0 any     anywhere             anywhere            

NAT:
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  561 75420 MASQUERADE  all  --  any    any     172.31.255.0/24     !172.31.255.0/24     

c) Create or update test environment:

Now we are ready to get the test environments running, autopkgtest
provides a helper script for that purpose:

 sudo autopkgtest-build-lxc debian sid

It will warn about a missing LVM volume "lxc", but this can be ignored, it
will work nonetheless.  This command will also be used to update the
environment, creating a new one with a temporary name which will replace
the old one, once it has been created successfully.

d) Testing

Running a test is now very easy. Build the package in question, and then
run:

 autopkgtest PACKAGENAME_0.0.0-0_ARCH.changes -- lxc -s -e autopkgtest-sid

This will run all the defined tests and create a report at the end.

Grüße,
Sven.