Bug#762700: systemd: journald fails to forward most boot messages to syslog

[Christian Seiler]

Am 2015-02-26 00:04, schrieb Michael Biebl:
>> Also: I'm not looking for a solution that solves this for any
>> possible amount of log messages, but I do think that the current
>> limit doesn't provide enough of a safety zone that one would like
>> to rely on.
>
> If we can make the journald -> syslog forwarding more robust, without
> introducing side-effects, by all means, let's do that. I'm happy to 
> take
> suggestions how to achieve that.

Well, see my suggestion to increase the max dgram queue length
(towards the end of [1]). Basically, call sysctl to increase it
before syslog.socket is started, I did that with a oneshot
service.

[1] 
http://lists.alioth.debian.org/pipermail/pkg-systemd-maintainers/2015-February/006197.html

>> w.r.t. imjournal: I wouldn't really care about configuration there,
>> but rsyslog is built without imjournal support
>
> That is not true, journal support has been enabled in rsyslog for 
> quite
> a while. It's split into a separate package called "rsyslog-journal",
> maybe that's why you didn't notice it?

$ apt-cache search rsyslog-journal
$ rmadison rsyslog-journal
$

(imjournal also yields nothing)

Am I doing something wrong?

>> And irrespective of all that, the problem of the boot messages
>> remains (and I don't mean only the really early ones, I mean
>> especially those of services ordered after $syslog under sysvinit).
>
> What specific problem are you talking about here?

http://lists.alioth.debian.org/pipermail/pkg-systemd-maintainers/2015-February/006207.html

I listed a couple of boot messages in there in response to your
question that do appear in the journal, and did appear in syslog
in Wheezy, but don't appear in the syslog in Jessie.

In case my explanation wasn't quite clear:

  - Wheezy or Jessie w/ sysvinit:
    Most non-early boot services are ordered after $syslog, which
    translates to rsyslog or syslog-ng init script. That means all
    messages they generate are stored in syslog.

  - Jessie w/ systemd:
    $syslog is equivalent to syslog.socket, so once
    remote-fs.target is reached, these services are immedaitely
    started. But syslog.service is also ordered after
    remote-fs.target, so it's started in parallel. syslog.service
    might take a while to start up, and other services are going
    to be faster. During that time, syslog.socket is still full
    with the first 11 early-boot messages, so messages generated
    by those services before syslog.service has started consuming
    the queued messages will be dropped from forwarding and not
    appear in syslog. This is clearly a regression compared to
    Wheezy. Not only very early boot messages get dropped, but
    also those that would have been logged under sysvinit.

    (Ok, technically some of those messages are from stderr of
    those daemons, so Wheezy wasn't quite as verbose, but
    still...)

    For example, if exim doesn't start up because of some weird
    temporary error, I would like to be able to figure that out
    once I notice it, which might not be immediately, depending
    on the circumstances. And if the journal has then already
    rotated the boot messages away, and it's not persistent, I
    won't be able to find out what happened.

Christian