[pkg-lxc-devel] Bug#860974: Bug#860974: lxc: systemd and other inotify-based programs fail with "Too many open files" with a large number of LXC containers

[Evgeni Golov]

Hi,

thanks for the report!

On Sun, Apr 23, 2017 at 12:31:02AM +0200, Baptiste Jonglez wrote:
> I am using LXC on stretch, running a number of stretch containers.
> 
> When running around 15 containers, systemd starts to fail, both on the host
> and inside containers:
> 
>     # journalctl -f
>     Failed to get journal fd: Too many open files
>     # systemctl stop nginx
>     Failed to allocate directory watch: Too many open files
> 
> The problem seems to be that systemd uses inotify quite a lot, and the limit
> on inotify listeners is very low by default:
> 
>     # sysctl fs.inotify.max_user_instances
>     fs.inotify.max_user_instances = 128
> 
> Increasing this value indeed solves the problem.  The issue was described
> in this blog post:
> 
>     https://kdecherf.com/blog/2015/09/12/systemd-and-the-fd-exhaustion/
> 
> Attached is a patch for lxc that configures this sysctl setting to a higher
> value.  I don't know if this is the "Debian way" or if some other solution
> is preferred.

Your patch looks quite similar to what was added in LXD in Ubuntu recently:
 https://github.com/lxc/lxd-pkg-ubuntu/commit/333e2671a91861b949a2b990459656a3b065e5fb
 https://bugs.launchpad.net/juju/+bug/1602192

I think we also should add something like
 sysctl -q -p /etc/sysctl.d/lxc.conf || true
to the configure target of postinst, to actually load the file directly,
instead of waiting for a reboot?

However, I am not 100% sure that packages should alter sysctl settings without
telling the admin.

Regards
Evgeni