Bug#823530: systemd 228 reduced maximum number of tasks in a cgroup to 512 by default

[Martin Pitt]

Hello all,

Sebastian Schmidt [2016-05-05 20:14 +0200]:
> Whilst debugging why Chrome would regularly fail to create new
> threads[1] and zsh couldn’t fork I noticed that:
> - - All affected processes run in the /system.slice/xdm.service cgroup
> - - “systemctl status xdm.service” says “Tasks: 476 (limit: 512)”

Antonio Terceiro [2016-05-06 15:45 -0300]:
> I am seeing some inconsistency in this. On my laptop, I don't see any
> limits being applied, even though I am on a fully up to date unstable
> (tried both with my main account from a terminal emulator in GNOME, and
> from a console login with a freshly created user account):
> 
> $ ulimit -u 22988

Note that these are two different things:

  * TasksMax is enforced via the pids cgroups, and only applies to
    .service units started by systemd. It should not apply to user
    sessions (i. e. .scopes) started through libpam-systemd, but
    as Sebastian is not using them all subprocesses of xdm.service
    inherit xdm's limit.

 * ulimit -u is the classic ulimit which is unrelated to TaskMax=
   (instead, it is LimitNPROC= if you want to set it in a unit).
   systemd does not have/impose a default setting for ulimits, this
   usually comes through /etc/security/limits.conf and is being
   applied to PAM sessions only, not to .services.

> I do appreciate the idea of systemd imposing sane limits on tasks, but
> this will cause A LOT of breakage in all sorts of places, and we need
> figure out a plan forward. Will every package need to start declaring
> higher limits explicitly?

Since we released Ubuntu 16.04 LTS with this I've also gotten reports
about this. E. g. MySQL and RabbitMQ easily break the TasksMax=512
limit even under moderate load.  (https://launchpad.net/bugs/1578080
is one example) Note that this actually limits threads, not just full
processes.

So in retrospect, having a default limit there was not such a good
idea after all: 512 is way too much for most "simple" services, and
it's way too little for others such as the ones mentioned above. There
is also no particular rationale about "512", so even if we'd bump it
to 1024 we'd just make the limit even less useful while still breaking
software.

So I think we should disable the default limit. It is both much safer
and also much more effective in terms of guarding against berserk
programs/bugs/unintended fork bombs etc. to set limits in units
individually. Once someone looks at one, this is then a great time to
also flip on the other resource and privilege limitations that systemd
offers, such as CapabilityBoundingSet=, SecureBits=, PrivateDevices=,
PrivateNetwork=, ProtectSystem=, ProtectHome=, etc.

So I'm proposing to change the default to "infinity". We could do this
with a /lib/systemd/system.conf.d/no-task-max.conf, but this would be
confusing to someone looking at /etc/systemd/system.conf, so a patch
might be preferable (i. e. reverting upstream commit 9ded9cd). I'll
also raise this upstream (once I have internet again), perhaps we can
even change the default there.

Opinions?

Thanks,

Martin
-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-systemd-maintainers/attachments/20160506/09db0688/attachment-0002.sig>