Bug#798625: systemd-networkd: Runs arbitrary inappropriate scripts on network changes

[Josh Triplett]

On Sun, Sep 20, 2015 at 04:08:51PM +0200, Michael Biebl wrote:
> Fwiw, I mentioned similar concerns as Josh when I discussed this topic
> with Martin but I also see where Martin is coming from.
> 
> The missing hook/extension mechanism in networkd is something which is
> an issue.

I wouldn't necessarily put it *that* way.  The functionality currently
handled by ifupdown hooks needs handling in some way; that doesn't mean
networkd needs to run arbitrary hooks.  That needs some very careful
thought before creating what amounts to a new API expectation.

And after careful review of all the hooks, this doesn't seem like a
particularly serious issue at all.

> Am 20.09.2015 um 15:54 schrieb Martin Pitt:
> >> Hardly a zillion packages; here's a complete list of Debian packages:
> >> > 
> >> > This seems quite manageable, especially once you rule out the ones that
> >> > make no sense at all with networkd (or with anything other than
> >> > ifupdown).  I'd be happy to help evaluate these further.
> > That would certainly be nice. It's not that I like the hooks much, but
> > I believe that nothing at all is gained by tightly coupling
> > transitions that way, like "sorry, you can't use networkd without
> > resolved and without fixing those ~ 50 packages first". I like the
> > evolutionary approach more than the revolutionary one.
> 
> I basically asked the same: how many packages do actually need such a
> hook and what do they need it for and would those be eligible for being
> run under networkd.
> 
> We didn't fully answer that, so I started extracting the scripts and
> having a look.
> 
> See https://people.debian.org/~biebl/ifupdown-hook/ and
> https://titanpad.com/P040gSeFyr
> 
> What I'm concerned about is, that our ifupdown hook support is
> incomplete and some hooks certainly should not be run under networkd.

Agreed.

> If we are going to provide a hook mechanism, maybe defining our own is
> better then an incomplete/incompatbile ifup.d hooks support.

At the very least, any such mechanism should go through systemd unit
files.  That would have several advantages: for instance, units that
want to run both periodically and "when the network comes up" can DTRT
(see .timer units with OnCalendar and Persistent, or with
OnUnitInactiveSec; policies like "once a day but only with the network
up" don't seem far-fetched here).  That would also provide the full
functionality from "man systemd.exec".

On top of that, any mechanism that tries to determine if the network is
"up" needs real connectivity detection, to deal with things like captive
portals, VPNs, and proxies.  I seem to recall upstream doing some work
on connectivity detection, and NetworkManager already supports that to
some degree (though it doesn't support policies like "don't ever route
traffic over this interface, *only* use it to run a VPN").

> Anyways, let's first figure out what packages actually need the hooks
> for, anyone who wants to help filling out the titanpad is welcome

Done; I've gone throuh the entire set of extracted hooks and documented
them all in the titanpad.

After reviewing them all, I *don't* think an arbitrary hook mechanism
makes sense here.

And if, in the short term, some users say "I can't migrate to networkd
yet, it doesn't support $foo", that seems fine; we certainly don't plan
to eliminate all other network configuration systems from Debian, and
eventually someone might decide to add support to networkd.  At the
moment, networkd still has some glaring omissions, most notably wireless
and VPN support, and I wouldn't expect any mass migration to networkd
until those get fixed.  And in the course of fixing those, networkd will
likely gain significant additional functionality, such as to cooperate
with a service that manages a VPN (since I don't think it makes sense
for networkd to reimplement things like openvpn, tinc, or iodine).

To summarize the existing ifupdown hooks, we have:

- Hooks that configure a network interface in ways networkd already
  knows how to handle natively.  (Examples: macchanger, bridge-utils,
  vlan, avahi-autoipd, wide-dhcpv6-client.)

- Hooks that configure a network interface in ways networkd needs to
  learn how to handle natively, in some cases cooperating with other
  system daemons.  (Examples: hostapd, openvswitch-switch, ucarp)

- Hooks that shouldn't exist at all.  (Examples: arping, avahi-daemon)

- A few instances of hooks for virtual machine network interfaces,
  setting up proxy ARP to make those interfaces reachable.  (Examples:
  uml-utilities, vzctl)  networkd already understands things similar to
  this (see the "vxlan" type in "man systemd.netdev", which has an
  ARPProxy key; also see the "tun" and "tap" support).  A few additional
  types and directives should cover the extra cases, along with
  documentation for users to migrate from /etc/network/interface
  directives to .netdev files.

- Daemons that manage a network tunnel/VPN/etc.  networkd already
  handles some of these directly, and needs to learn to handle the rest.
  See "man systemd.netdev".

- resolvconf (and its fork openresolv).  To the extent we can't just say
  "use resolved instead", this needs native support in either networkd
  or resolvconf.  Personally I'd prefer the latter.  Note that
  NetworkManager has native support for resolvconf.  In any case,
  resolv.conf files have fundamental limitations that will prevent them
  from ever supporting more complex configurations, such as split DNS;
  resolved can and should learn to handle those cases.

- Firewall scripts.  (Examples: shorewall-init, uruk.)  In some ways,
  this seems rather fundamentally broken, in that the interface will
  come up before the firewall; any sensible firewall implementation
  needs to have the rules in place *before* the interface will pass
  packets.  Common cases should perhaps have "native" support in
  networkd.  Otherwise, these just seem like special cases of one-shot
  programs.

- Hooks to run a one-shot program that requires a network connection
  when an interface comes up.  (Examples: ntpdate, slrn, slrnpull,
  clamav-freshclam.)  Note that most of these programs should also run
  periodically *while* the network remains up; they also need to handle
  issues like connectivity detection, which means they still can't
  assume a working network when run.  In some cases, the right solution
  involves replacing one-time programs with daemons that handle dynamic
  networking (ntpdate -> timesyncd or ntpd; run freshclam in daemon
  mode).  Failing that, some kind of common helper program/daemon might
  make sense here.  Also, some of these might make more sense inside a
  user session rather than systemwide.

- Many many hooks that kick a daemon with reload, force-reload, restart,
  or a daemon-specific mechanism every time an interface comes up (or
  sometimes goes down).  Most of these look like daemons that don't know
  how to handle dynamic network configuration.  The right long-term fix
  involves fixing those daemons (which normally shouldn't be a
  monumental task; see my previous mail about how to fix sshd, which
  also applies to things like lprng or epoptes-client).  As for mail
  servers like postfix and sendmail, don't mail servers normally run the
  queue quite frequently anyway, with defaults in the "every few
  minutes" range?