Bug#762700: systemd: journald fails to forward most boot messages to syslog

[Christian Seiler]

Am 2015-03-03 15:33, schrieb Michael Biebl:
> Am 02.03.2015 um 15:42 schrieb Christian Seiler:
>
>>  - SOLUTION (mostly the same as before, but SendBuffer is set on a
>>    different unit and a Condition is added to the service):
>>
>>       1. Increase max_dgram_qlen to a reasonable value. The easiest
>>          way is via a systemd service:
>>
>>          /lib/systemd/system/systemd-setup-dgram-qlen.service:
>>          [Unit]
>>          DefaultDependencies=no
>>          Before=syslog.socket
>>          ConditionPathIsReadWrite=/proc/sys/net/unix/max_dgram_qlen
>>
>>          [Service]
>>          Type=oneshot
>>          ExecStart=/sbin/sysctl -w net.unix.max_dgram_qlen=512
>>          StandardOutput=null
>
> A proper Description= is missing and it probably should have
> RemainAfterExit=yes.
> That said, why not simply use a sysctl.d conf snippet?

systemd-sysctl.service is not ordered before the socket. Also it's
ordered after a bunch of services, and syslog.socket is typically
available really early.

>> I've been using this on a Jessie system for the past couple of days
>> (well, drop-ins in /etc, actually), and I didn't notice any
>> side-effects so far.
>
> Did you run it with 1. and 2. applied, or only 1.?

Only 1. so far. And 2. may not be necessary.

Full disclosure: I decided to try to also tackle this issue on the
kernel side (not for Jessie, but in general) by writing a patch that
allows the user to change that option for individual sockets.[1] One of
the issues that were mentioned in replies so far was that due to file
descriptor passing in UNIX sockets, increasing the limit could reach
OOM a little easier if an attacker were to do weird things. (Not with
the syslog socket, but with a socket of their own.) I would argue that
the current limits to the number of file descriptors etc. in a typical
distro don't even come close to preventing OOM anyway, so it doesn't
hurt much, but it's only fair to share this before you change anything.

[1] <http://www.spinics.net/lists/netdev/msg318492.html>

That said, w.r.t. buffer sizes: I did some tests during the test of my
kernel patch, and if each datagram is less than 192 bytes in size
(likely true for log messages), there will fit 555 of them in the
standard send buffer of a socket - so a limit of 512 (513 in reality,
the kernel is off by one) would cap that beforehand. (1 byte or 192
bytes don't make a difference btw., only >= 193 bytes the number
decreases.) So yes, setting SendBuffer on dev-log.socket is probably
not needed, unless you have really, really large log messages. (The
reason I mentioned the buffer in the first place was the fact that
upstream sets it AND for some rather long test messages, I hit the
buffer limit before I hit the queue length limit, but those are
probably unrealistic.)

So if the sysctl is reliably set before the socket is created, we
should be good, the buffer size change is probably not needed unless
there are REALLY long log messages.

Christian