[Pkg-systemd-maintainers] Bug#734951: Bug#734951: systemd: somehow starts LSB stuff in the wrong order

Tue Jan 14 21:53:53 GMT 2014

Hey Uoti and Michael.

>Something on your system is explicitly stopping fail2ban, by calling
>"systemctl stop fail2ban.service" or something which maps to that such
>as invoke-rc.d. Under sysvinit stopping the service before it has
>actually been started would have no noticeable effect, but under systemd
>it cancels the queued start action.
Ah I see... and actually you're right...
What I do is about the following:

Since my iptables rules provide "hooks" (i.e. dummy rules), which
fail2ban should replace with its own when started ... AND reverted to be
hooks when stopped, I've also changed the iptables-persistent script to
stop/start fail2ban, when iptables-persistent is
stopped/started/restarted.
(Obviously a hack, but not really possible to do that better in sysinit,
I guess.)

I wasn't aware that this would have a queueing effect.... (and
admittedly I've totally forgot about it).

Now... is there any clean way to do that with fail2ban/iptables
persistent.. i.e. that when iptables-persistent is stopped/restarted,
systemd automatically stops/restarts fail2ban as well?
Especially since both are not yet converted to systemd.

>In general, if you want to make sure that a service cannot start unless
>another has been successfully started, add the "Requires" or "Requisite"
>field to the unit.
But AFAIU, this would be like that every service for, which iptables
rules MUST be in place due to e.g. the site's local security
constraints, would need to Requires/Requisite iptables-persistent (or
something similar).
Realistically,.. it's not going to happen that all maintainers add
something like this to their unit files (once they convert).
Isn't there something like what we've had in sysvinit... e.g. that you
reverse-depend on $networking?

On Tue, 2014-01-14 at 21:28 +0100, Michael Stapelberg wrote:
> In general, the sysvinit configuration source works fairly well in
> systemd, but there are good reasons why we want to migrate to systemd
> entirely and not live with the compatibility layer forever. Edge cases
> such as this one are such reasons :).
sure... but yet we have #727708 bothering us ;-)

> Now this is interesting. Something is communicating with systemd and
> tells it to stop fail2ban. This is the reason you first see the
> “stopped” message. I don’t really know what the thing is that talks to
> systemd here, but my guess is that it’s some sort of ifup.d-hook or
> something like that. The iptables-persistent package doesn’t ship
> anything like that, so maybe it’s a local system modification of yours?
Yeah... see above :-)
I didn't imagine that systemd tries to be that smart, and also
"converts" calls of /etc/init.d/xx stop to be queued.

I haven't thought that through... but wouldn't the compatibility to
legacy LSB script be better,... if it doesn't try to be that smart?

> I don’t think that’s true, but I have worked with systemd far more than
> with sysvinit, so I might be biased. Apart from the extensive logfile
> that systemd can produce, there are tools to generate graphs out of the
> boot order and other simulation tools (e.g. systemd --test). Obviously,
> none of this can predict what actually happens _outside of systemd_,
> such as the stop request systemd receives in your logfile.
Okay maybe I explained it wrong:
I'm absolutely sure that units can be configured in a way, that it is at
least as secured as with sysvinit that e.g. iptables is loaded before
network-critical things run.

The question though is: are our units in Debian set up this way... and
if not what's actually the best way do get that behaviour.

I mean it's not only iptables. Imagine services like haveged or any
similar entropy gathering daemon (ekeyed and friends).
One basically wants that these guys are initialised in the very
beginning, ideally as early as possible... just like you want to have
the random seed read early in boot and fed to the PRNG (which AFAIU is
done by systemd in C code, right?).

For "services" like iptables-persistent or similar firewall programs,
people would usually expect that the rules are in place before anything
except perhaps the loopback network interface is brought up.
And I think this should not be like that other services as Apache httpd,
or postgresql depend on iptables-persistent.service... since people WILL
forget this.
I think some general solution must be found, like e.g. having it in the
ifup generators... but is that enough? What if some other tool like NM
does playing around with interfaces.

For services like OpenVPN/IPsec... well I think it's also "important" to
have them already running _before_ any other services do networking...
but I also think it's not as critical as with e.g.
iptables-persistent,... since when it's really that critical for a site
to have certain routes secured, they should probably have this reflected
y according netfilter rules.

Actually I think that this is at least partially also the responsibility
of the systemd maintainers in Debian,... since you guys have probably
the most experience how to do this kinds of things cleanly.

So is there anything going on in that direction?

I mean as you can see... it's not only iptables... but basically all
things which will be activated by systemd anyway (something like haveged
is typically no dynamically pulled in by services that think they need
more entropy)... but where one wants or even need them to be started
very early before "normal" daemons.

Oh and I do not claim, that this has been done perfectly in sysvinit
where e.g. haveged starts more or less at any time (it just depends on
$remote_fs)... but this doesn't mean we can't do such things better with
systemd.

> How about using pre-up iptables-restore < /etc/network/iptables in
> /etc/network/interfaces then? :)
I'm not sure whether that works with all rules... especially when you
have rules that use ifaces which are then not yet up or perhaps not even
existing.

> I do agree that network config is a bit unsatisfactory in general,
> though. An effective and mostly parallel boot only highlights this
> problem, but doesn’t create it, IMO.
Well at least we need to eventually find some securely working solution
for that,... cause this is really security critical.

> Can’t really talk much about this, since it has been many years since I
> last touched ipsec and I don’t know of anyone who uses it these days.
Well it's the same for OpenVPN... or similar.
And yes,... IPsec is quite alive ;-)

Cheers,
Chris.