Bug#815020: breaks coredump handling for systems without systemd-coredump

[Wouter Verhelst]

On Wed, Feb 17, 2016 at 06:33:59PM -0300, Felipe Sateler wrote:
> On 17 February 2016 at 13:12, Wouter Verhelst <wouter at debian.org> wrote:
> > Package: systemd
> > Version: 229-1
> >
> > Hi,
> >
> > systemd contains the following line (src/core/main.c):
> >
> >                         (void) write_string_file("/proc/sys/kernel/core_pattern", "|/bin/false", 0);
> >
> > The intent is that at some later point, systemd-coredump is started,
> > which then sets it to something more useful.
> 
> Note that "systemd-coredump is started" means exactly "set it to
> something more useful" systemd-coredump does not run as a daemon, but
> rather is invoked by the kernel.

That's not what the NEWS file seems to suggest, but I'll take your word
for it.

> > However, in Debian, systemd-coredump is in a separate package that
> > systemd itself does not depend on. As a result, if systemd is used as
> > PID1, but systemd-coredump is not installed, all core dumps are thrown
> > into the bit bucket; not just those that are generated during bootup,
> > but all core dumps after boot, too.
> >
> > They're still generated, however, since systemd also sets the default
> > soft resource limit on core file size ("ulimit -c") to the "unlimited"
> > value, rather than having the hard limit be unlimited and the soft limit
> > be set at 0.
> 
> Apparently, if the core_pattern is a program, then the limit is
> ignored by linux, and should instead be honored by the called program.

yes, well, so they're generated at any rate, even if the limit is set
:-)

> > This is not an ideal situation:
> > - Generating a core dump, especially in case of a program that uses a
> >   lot of memory at the time where the dump is generated, requires some
> >   processing. If the core dump is thrown away, this is wasted processing
> >   time. If the system is loaded (e.g., because something is forking and
> >   segfaulting a lot), this is problematic behaviour. By setting the soft
> >   core file size resource limit to 0, core dumps aren't even generated,
> >   improving performance.
> 
> How much resources does this actually consume? If the receiving
> process does nothing, I don't think there would be much effect going
> on.
> 
> According to this (anectodic) upstream list post, the cost is negligible
> 
> http://article.gmane.org/gmane.comp.sysutils.systemd.devel/31244/match=core_pattern

It may be "negligible", it won't be "nothing". If the system is under
high load, thousands of "negligible" isn't that any longer.

> > - The kernel.core_pattern sysctl setting is a system-wide setting; it is
> >   not possible to modify this other than by using CAP_SYS_ADMIN (IIUC)
> >   permissions. In contrast, the resource limits are per-process
> >   configuration; any process can change its resource limits at will.
> >   This allows the system to have a default of "no core dumps", but at
> >   the same time, allows a developer or a system administrator to
> >   temporarily enable core dumps for a particular process, without
> >   requiring administrator capabilities.
> 
> Because systemd-coredump respects rlimit, if systemd did not change
> the rlimit then it wouldn't work by default: meaning that
> systemd-coredump would have to somehow tell systemd (maybe at install
> time) to change the default rlimit to something saner, but without
> interfering with any admin changes.
> 
> Because systemd conf snippets are always read after the main
> configuration file, it means that systemd-coredump cannot ship a
> system.conf snippet altering the default rlimit, because it would
> override admin changes in system.conf.

I'm not sure I understand this bit. Can you clarify?

> > I'm sure systemd-coredump replicates some of the above, but it isn't
> > always the best choice to have a core dump be piped into a process, and
> > therefore using the traditional "just dump everything to disk" logic can
> > still be beneficial.
> 
> It can still be restored via a sysctl.d snippet, and the rlimit via system.conf.
> 
> Maybe this should be documented somewhere on upgrades?

Possibly.

An alternative solution would be that systemd-sysv shipped with a unit
which would set the core_pattern back to default, which could be
overridden or disabled by a unit shipped by a unit shipped with
systemd-coredump? That would be less surprising -- I have to say I spent
a long time tracking down what happened.

-- 
< ron> I mean, the main *practical* problem with C++, is there's like a dozen
       people in the world who think they really understand all of its rules,
       and pretty much all of them are just lying to themselves too.
 -- #debian-devel, OFTC, 2016-02-12