systemd and "passive" security dependencies for services?

[Christoph Anton Mitterer]

On Thu, 2014-05-22 at 16:40 +0200, Tollef Fog Heen wrote: 
> $network is translated to network.target for LSB services, so it is in
> fact the same thing.
Sure... 

> You're pointing out that «the network is up» is ill-defined.  That is a
> correct observation and technically never true in a dependency system, since it might change at
> any point.  However, that's not a very useful definition and so systemd
> provides it for software that doesn't know how to deal with dynamic
> networks.
Sure... but that's how it is... I guess it's absolutely sure that there
will always be software, which doesn't use e.g. NM (and given the issues
I have with NM, I'm actually happy about that).
But even if all software _would_ use NM (or something similar)... NM
doesn't take care about e.g. netfilter rules.

And as you said... it's not really the best "design" for services to
depend on something like networking... as it's rather some legacy thingy
which might include anything or nothing.

> Just the same as today, it's ill-defined.  Does it mean you can bind to
> an interface?  Other than localhost?  Link is established?  IP address
> set?  You can bind to all the interfaces?  You can talk to the default
> gateway?  Your routing table has somewhat converged and you can speak to
> most of the internet?  Something else?
Sure... and don't get me wrong... I never tried to show that systemd is
worse than sysvinit.

The only "advantage" that we had with sysvinit... things were that
"legacy"... and all networked services usually depended on $network...
that from a security POV we got the "netfilter rules already loaded" for
free and "for sure".
It wasn't a "hard" for sure... if some init-script would have forgotten
to depend on $network,... then same situation... the service might have
gotten loaded before netfilter rules were in place.
Neither was there any guarantee that the rules were successfully
loaded... just that they were tried to be loaded before the service.


> > - it's a passive unit, so AFAIU it's not to be pulled in by the services
> > consuming that functionallity (i.e. webserver) but rather the provider)?
> 
> I don't know what you mean by passive unit.
Well the upstream documentation I've referenced uses that phrases...
claiming network.target was passive, while network-online.target was
active.


> > - it's more about networking, less about netfilter,... so I think there
> > are services for which iptables need to be set up before they start
> > running, but where the interfaces, DNS, etc. don't need to work yet (or
> > at all, if there is just networking on the loopback)
> 
> loopback works before any services are started, so that you don't have
> to worry about.
Sure... what I meant was rather:
Even though it's probably highly specialised... there may be setups
which require the netfilter rules to beloaded even before the loopback
is up.
Though admittedly I haven't seen such...


> There's nothing in the LSB definition that says «firewalls are up».  If
> you want that, have a ferm.service (or iptables-persistent.service if
> you prefer that) that has Before=network.target and it will run before
> you have «useful networking».
Okay but I though network.target itself is rather something legacy... is
it really like "policy" that all services that do networking depend on
that?


> > - there seem to be only few services that do networking which actually
> > somehow depend on it...
> Services are hopefully rewritten to not having to rely on it.
That's one of the problems...
It's good if a service doesn't depend (i.e. the daemon doesn't crash or
whatever) if networking isn't there.
Therefore most services would only say:
Wants=networking.target

But if we do like you say above (i.e. iptables-persistent has
Before=network.target)... then we have no real guarantee that the rules
are up, when the service that Wants=networking.target starts.




> > So I guess one thing must be guaranteed... if one uses e.g.
> > iptables-persitent to load the iptables rules,.. than seervices that do
> > some networking must start after it,... and ONLY if iptables-persitent
> > was loaded successfully.
> 
> you need the interface not to be brought up before you apply the
> iptables rules.
Sure.. I didn't claim that, did I?
Actually the rules must be loaded BEFORE the ifaces are brougt up.


> If iptables-persistent wants to run before network.target, it should
> declare that by doing WantedBy=network.target and Before=network.target
Wants/WantedBy is not enough here...
According to the docs:
Wants= 
           A weaker version of Requires=. A unit listed in this option will be
           started if the configuring unit is. However, if the listed unit
           fails to start up or cannot be added to the transaction this has no
           impact on the validity of the transaction as a whole. 

So  that means loading iptables could fail, but the services would still
start => security issue


> > I think best would be if there were some more such special targets like
> > network-security or so...
> > All services that do some networking should Require=/After= that... and
> > that target in place should pull in those of the installed services like
> > iptables-persistent...
> Why should security be a bolt-on?
Well I guess everyone being only somewhat security conscious will agree
that it's better to not have some service started at all (which will
usually be noted immediately) than to have it started unsecured (which
usually noone notices that soon).



Can we agree on the following:

1) daemons (postfix, postgresql, httpd, gdm3) but also the system as a
whole requires some "things" with respect to security being present
(let's take again "loaded netfilter rules", which is not only important
when you run listening services but also on normal desktops, i.e. when
you just have login sessions).


2) These "things" must be in place, _before_ the service is used... e.g.
netfilter rules must be loaded, _before_ the daemons start, _before_ the
user could do any networking in his sessions, etc. ... or e.g. a TRNG
must be activated, _before_ services want to use high quality entropy.

If not, then one might have a short time frame, where e.g. postfix
listens and networking is not secured... or where httpd makes TLS
connections, but it still uses poor entropy from right after the system
boot.


3) In systemd, we only want to start things when necessary... and also
only want to pull in the least necessary stuff... i.e. if postfix
auto-discovers new interfaces that go up and starts listening on them...
we don't want it to depend on network.target (which already expects all
the ifaces to be up)... but we DO want it to depend on "netfilter rules
loaded".
And having the netfilter rules loaded as an even (e.g. just before a new
interface is brought up) just doesn't work in practise.
So neither can we delegate that task to postfix, or to the
program/service that brings up the interface (e.g. ifupdown, or NM).. we
have to do it in the system start.


4) The way this (i.e. how/when e.g. netfilter rules are loaded, and how
other units depend on that) is done should be mostly common for all
stuff in debian.
i.e. it shouldn't be that case that postfix Wants
+After=iptables-persistent, while httpd Wants+After=ferm and perhaps
ejabberd simply expects that all stuff was already pulled in by
basic.target
So I guess the systemd team should come up with kind of a policy of
best-current-practices on all that.

Not only does this teach the admins/maintainer the "right way" to do
it,.. but it also avoids situations where a maintainer could easily
forget about such (security) dependencies.

E.g. there is /lib/systemd/system/cups.service
and as far as my (perhaps limited) understanding tells me... that
doesn't depend directly or indirectly on iptables-persistent|ferm|etc.
(not even on network.target).
Of course it implicitly depends on basic.target... but that in turn
doesn't pull in iptables-persistent|etc.
cups may listen on the network, so we already have the situation that a
configuration might depend on security by having iptables rules in
place.

So if I didn't miss anything in that example,.. one sees how easy it can
happen, that security is missed out.


Now if I'm right so far, I guess two questions come up for each of such
"things" like iptables/TRNG/etc.
a) When and how (i.e. via which units) should it be pulled in?
b) What should happen, if the "thing" couldn't be started by systemd or
if the daemon (if any) that runs "thing" fails/crashes.


In my opinion, security always comes first, so I rather think that if
e.g. iptables can't be loaded (perhaps because I did a typo) then my
deamons should start neither.

It's more tricky to answer, what happens when I reload a "thing"that
fails... should the services that depend on it be stopped then?
With iptables things are still easy,.. if the rules cannot be loaded,
the old ones stay in place. But what if e.g. my TRNG entropy daemon
fails... should httpd be stopped then? Tricky question... I'd say per
default it should not stop... but it should be easy for admins to find
out, how they get just that behaviour.

So I guess in the end it depends on the "thing" on what we should do as
BCP.




Let me thing how I might handle that for some of  "things" that are most
important to myself at the moment:


I) loading of the random seed file
I think this is important for every program, not just some services or
daemons,... so it should happen always by some units that are not just
pulled in by services like gdm3 or postfix.

Right now systemd-random-seed-load.service is (amongst others):
Before=sysinit.target
DefaultDependencies=no

And sysinit.target
Wants=systemd-random-seed-load.service
via the
symlink /lib/systemd/system/sysinit.target.wants/systemd-random-seed-load.service.

So question (a) from above is fulfilled... everything's fine, since
sysinit.target is defined to be very early in the boot (even before
basic.target)... so it will be before all usual suspects like
postfix/postgresql/httpd and also before the user can log in an run any
programs. 
I see however that right now it may run in parallel with (on my system) at least these, which are the After= of sysinit.target:after:
console-setup.service(exited), cryptsetup.target(active), debian-fixup.service(dead), dev-hugepages.mount(mounted), dev-mqueue.mount(mounted), ebtables.service(exited), emergency.service(dead), emergency.target(dead), etc-setserial.service(exited), hdparm.service(exited), ifscheme.service(exited), iptables-persistent.service(exited), kbd.service(exited), keyboard-setup.service(exited), keymap.service(exited), local-fs.target(active), mdadm-raid.service(exited), networking.service(exited), nfs-common.service(exited), proc-sys-fs-binfmt_misc.automount(running), qemu-system-x86.service(exited), quota.service(exited), rpcbind.service(running), setkey.service(exited), setserial.service(exited), swap.target(active), sys-fs-fuse-connections.mount(mounted), sys-kernel-config.mount(dead), sys-kernel-debug.mount(mounted), systemd-binfmt.service(dead), systemd-journald.service(running), systemd-modules-load.service(failed), systemd-random-seed-load.service(dead), systemd-readahead-collect.service(dead), systemd-readahead-replay.service(dead), systemd-sysctl.service(exited), systemd-tmpfiles-clean.service(dead), systemd-tmpfiles-setup-dev.service(exited), systemd-tmpfiles-setup.service(exited), systemd-udev-settle.service(exited), systemd-udev-trigger.service(exited), systemd-udevd.service(running), zfs-fuse.service(exited)
Has someone really checked whether nothing of these would benefit from a
seed /dev/urandom? Having a short glance I'd say it's okay.

Question (b)... should anything fail if the seed can't be loaded?
Well I guess it should do so, because an unseeded /dev/urandom can be
really a big security issue... but OTOH,... what to do with fresh
installations (where there is no seed file)? And do we really need to
make that sure by a:
sysinit.target: Requires=systemd-random-seed-load.service
?
There should be no situation that can easily arise (e.g. as simple as a
typo in my iptables rules file) which prevents the loading of the seed
file... and if someone screws up his system that much, than well... if
yourself shoot in your own foot than noone can help you.

So I'm a bit unsure... with respect to security and clean design I'd
say... Require= it... and work around the issues that arise from that...
but in practise it's probably more problems than advantages... and
unlikely to happen that loading the seed file fails.


II) loading netfilter rules
The first tricky thing here is, that we have many programs
(iptables-persistent, ferm, shorewall, etc.) that may do that job and
most of them don't even conflict each other.

Moreover... we have programs that may dynamically change the netfilter
rules and that may need to do something when rules loading is
started/stopped/reloaded.
E.g. I have some more complex iptables rules, where fail2ban doesn't
just append his hooks to the INPUT table, but where there is a specific
place denoted by some dummy rule where fail2ban hooks in and out when it
starts or stops.
So everytime I readload iptables-persistent... it fail2ban needs to be
stopped first and started afterwards again.
Of course it would be great if that could be somehow automated with the
awesome powers of systemd ;-)


Question (a) When/how should netfilter rules be loaded?
Definitely before any networking can be done, which I guess means before
any iface comes up (well we could talk about making an exception for
loopback interfaces).
So we need that to happen before both, any daemons that e.g. listen on
ports are run ... and before any "user" programs that do networking may
be started.
Also, just because we want to have netfilter rules loaded,... this
doesn't mean that all (or even any) interfaces should also need to be
brought up with that.

I guess it's not enough if we make the units that may bring up
interfaces having some Wants/Requires= and After= on our netfilter rules
loader... since if nothing of these units are run, the user could still
manually bring up interfaces and then no rules would be in place yet.
So I'd guess we should put that as well to some special target like
sysinit.target,... to the earliest one possible I guess (would that be
sysinit.target?)

Question (b)... well that's tricky now.. I personally definitely
wouldn't want to have any daemon that listens on the network started if
loading my netfilter rules has failed.
So that would mean we need a Requires= .
But OTOH it's probably bad if the boot process breaks just because of a
typo in some rules file,... and because sysinit.target Requires= it.
I guess I could live quite well with the system coming up (and the admin
being able to do repair works) without iptables rules in place.
But services like postfix should definitely not start... whether normal
user logons should be made possible... well... tricky question... it's
not that uncommon that admins block some ports where users could stumble
over security related stuff...

So can we perhaps have about the following:
A target like "network-secured" (or some other name)... which basically
has the meaning to pull in everything that is needed to secure
networking, BEFORE networking is brought up (i.e. that wouldn't include
services like IPsec, VPN).
This target should then be Wanted+After= by the early boot process (e.g.
by sysinit.target).
Further, daemons like postfix/httpd/etc. should Require+After= this
target.
All units like iptables-persistent|ferm|shorewall|etc. should be
RequiredBy+Before=network-secured.target

So AFAIU, that would give us the following:
Before the early boot sysinit.target,... and of the installed
iptables-persistent|ferm|shorewall|etc. are run... but the boot process
continues if the should fail for whatever reason, since sysinit.target
only Wants=network-secured.target
But once the daemons would start up... things change... if
network-secured.target failed because any(!) of the installed
iptables-persistent|ferm|shorewall|etc. failed... it would not start up,
since daemons should have a Requires=network-secured.target


I guess that's about the basic idea hope it works like that ;-)
So the really important thing now is, that such behaviours are made BCP
by you systemd maintainers... or otherwise there is little chance that
one will see widespread and homogeneous use of something like that over
all packages.

[Actually I always have some slight hopes that you guys find some time
to also look over the unit files once packages get systemd support and
to see whether it's just some plain conversion from the sysvinit script
to systemd, or whether it really uses all nice features of systemd that
would make sense for the respective package (well at least I guess that
not all package maintainers are systemd geeks as well ;-) ).]

Now obviously there are many questions left... I wrote all the above
mainly from the view of iptables-persistent (which is now actually
netfilter-persistent ^^).
I think that only needs to have /usr and /etc in place... and has no
more deps,... but I don't know about the others like ferm|shorewall
etc..
Some of them may do more than just loading the rules, and may therefore
have more dependencies which might mean one couldn't place a
"network-secured.target" that early during boot.

Or questions like "what happens if that e.g. iptables-persistent while
it runs" or "if it's manually stopped".
Well the former can't happen with iptables-persistent... since it's not
a continuously running dameon... but what should happen when one
manually stops it?
Of course, if an admin does that, he hopefully knows what he's doing,
and daemons like postfix/etc. should continue to run.

How do we handle network security "things" that run after ifaces are
brought up? Like VPNs or IPsec (actually I don't know, whether it's
necessary to have ifaces up for these to be configured, but I guess so).
I handle that via netfilter rules as well (i.e. I forbid any traffic
that is non IPsec'ed, which I consider to be security critical).
But for some people it may be useful to have a "network-secure-foo"
which is pulled in after ifaces are up (of course again the problem with
"which ifaces does that include?")... and which could pull in things
like strongwan.

But I guess this is a rather hacky thing and we should try to avoid it.
If you need IPsec/VPN for security,... set that up in your netfilter
rules (it's easy).
If you need DNSSEC,.. simply don't add any resolvers that don't
validate.


III) What about "passive" security programs like rhkunter|aide|samhain|
tripwire etc. pp.?
"passive" in the sense that other services have no direct technical
dependencies on them, but people may want them to be in place for some
services for security reason.
E.g. some paranoid admin may not want to have httpd ever started if aide
is not running, or if some daemon that constantly moves httpd's logs to
a safe remote machine is not running.
Such people may choose that if httpd should be rather killed instead of
continuing to run, when those programs fail.
=> Well I guess unlike the "netfilter rules loading" thingy, which is
very common and basic,... we cannot cover all these scenarios, and
people can simply edit there unit files if they want to add such highly
specialised stuff.
Also, since all these scenarious are so different in how they'd work, I
think it doesn't makes sense to provide a framework/BCP as I proposed it
with network-secured.target above.


IV) What about entropy gathering programs, e.g. haveged, audioentropyd,
ekeyd, egd, and so on.
In principle these are of the same category as III, but I think all of
them have a lot in common so it may make sense to provide some basic
framework/BCP as well.
It could be similar to the one for netfilter rules.
Having a "entropy-service.target" which is haveged|ekeyd|etc. are
RequiredBy=
Have that entropy-services.target being Wanted= as soon as possible in
the boot process...
In contrast to netfilter rules, one may however want to let it up to the
admin, to manally let e.g. httpd.service Require= that
entropy-services.target ... not sure about that.

Also, is there some way in systemd, that if one of haveged|ekeyd|etc.
would crash, that everything that has some kind of hard dependency on
"entropy-service.target" is immediately stopped?
Then admins could configure their e.g. httpd to rather stop, when it's
no longer guaranteed that "high quality entropy" is available.



Cheers,
Chris.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5165 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-systemd-maintainers/attachments/20140523/ed4adf6e/attachment-0002.bin>