PEP 3143 and customizations

[Helmut Grohne]

On Wed, May 21, 2014 at 02:55:34PM -0700, Ethan Furman wrote:
> Ben, how amenable are you to API change?

If API changes (or addition of a new class) are under consideration, I'd
like to propose the following use case:

do_some_setup_before_daemonizing()
with daemon_class() as finished:
    some_more_setup_in_the_child()
    finished()
    enter_main_loop()

In this example the __enter__ method calls open(exitparent=False) and
returns self.finished rather than self. Thus calling the returned
finished callable becomes mandatory. Now the __exit__ method checks
whether finished was already called and whether an exception occured:

 * finished was called: 
   The daemon terminated (no exception) or crashed (exception occurred).
   The parent is already dead. Reap the PID file as previously done.
 * finished not called, no exception:
   A programming error. Make the parent exit with a suitable failure
   message.
 * finished not called, exception occurred:
   An error happened during setup. Capture the exception details (+
   traceback?) and make the parent print this information prior to
   exiting with a failure.
 * (Optionally) finished not called within a reasonable timeout:
   Either a programming error or a deadlocking setup code. Terminating
   the parent with an error is probably the best thing to do here.

The second last point is where the proposed API change becomes really
useful.  One just lets errors prior to calling finished() pass knowing
that they will be shown in the parent rather than being swallowed in a
successful exit.

A failure in PID file creation should use the very same mechanism to
report the error.

>From my POV not being able to meet the expectation of systemd
Type=forking services with PEP3143 makes this PEP utterly useless. You
simply cannot use these components together. And even though you can use
PEP3143 services with other init systems, there are swallowed failures
and race conditions. PEP3143 makes it very hard to do the right thing.

Here are a few users of daemon that are doing it wrong:

http://sources.debian.net/src/nss-pam-ldapd/0.8.14-1/pynslcd/pynslcd.py
swallowing errors after daemonizing
http://sources.debian.net/src/bcfg2/1.3.4-1/src/lib/Bcfg2/Reporting/Collector.py
trying to print PID file creation failures to /dev/null
http://sources.debian.net/src/ftp-cloudfs/0.25.2+20140217+git2a90c1a2eb-1/ftpcloudfs/main.py
swallowing (PID file creation) errors after daemonizing
http://sources.debian.net/src/calypso/1.4/calypso.py
swallowing (PID file creation) errors after daemonizing

It's not like this is an isolated problem. Look at the harm already
done. Breaking API looks cheap to me when compared to this zoo of subtle
errors, that make failures go unnoticed.

Another bonus point of changing the API to the above is that it suddenly
becomes trivial to implement non-forking services with readiness
notification for systemd Type=notify. One just has to replace the
finished() method with something that does the notification back to
systemd.

If all else fails, please make turn detach_process_context into a method
of DaemonContext, so I can override it with something sensible rather
than replacing open() in full.

Helmut