order of operations when daemonizing

[Ethan Furman]

On 05/20/2014 05:33 PM, Ben Finney wrote:
> On 20-May-2014, Ethan Furman wrote:
>>
>> 1) Why are DaemonRunner and DaemonContext two different things?  I would have
>>     thought that part of being a "well-behaved daemon" included stopping and
>>     restarting.
>
> See the question “How can I run a service communicating with a
> separate daemon process?” in the FAQ document (‘doc/FAQ’ in the source
> tree).
>
> The daemon should normally run as an unprivileged user. So the service
> runner – which is commonly some part of the OS like the SysV Init or
> the SystemD service runner, but could alternatively be the daemon
> program running as root before opening the DaemonContext – needs to
> take care of starting and stopping and handling a PID file, and other
> privileged operations. So those need to be separate processes.

The issue with the PID file is that DaemonContext.open() does not create it until /after/ privilege has been dropped 
(that's when it calls `self.pidfile.__enter__`, which calls `pidfile.acquire`, which creates the file and writes the PID 
to it).

If the pid file is acquired before detaching the wrong PID is written to the file, but if privilege is dropped before 
acquiring then the file cannot be created.


>> 2) Why does DaemonContext lose privilege and detach before setup is
>> complete?
>
> Because that's an essential part of the contract. On successful return
> from ‘DaemonContext.open’, the process is running as a well-behaved
> daemon. It's an atomic operation.

Sorry, I meant the setup that ContextDaemon.open() does.  See above.
>
>>     a) I can't create a pid file in /var/run
>
> Essentially, creating the PID file and socket files and log files,
> etc. is the job of whatever privileged process starts the daemon
> (which may be the same program in the time span before it drops root
> privilege). Those files are then passed to the DaemonContext.open
> call.

As I stated above, the PID file cannot be created before the .open() call or the wrong PID is written to it.  For other 
files (log, socket, etc.), sure, open them before and pass them to `files_preserve` option.


> The API for ensuring a file remains open is the ‘files_preserve’
> option. The PID file is an excellent candidate for that option.

Well, not really -- at least not with the pidfiles available in lockfile, nor daemon.pidfile.  I had to enhance both 
daemon.pidfile.TimeoutPIDLockFile and daemon.DaemonContext.open() to get the pre-create + later write pid behavior to 
make this work.


> This topic is a source of Frequently Asked Questions, not only about
> ‘python-daemon’ but about daemons in general. There are multiple,
> conflicting, conventions around how to handle PID files; so the
> ‘python-daemon’ library delegates the decision to the programmer.
>
> If someone could write an entry for the FAQ document I'd appreciate
> it.
>
>>     b) I don't see any errors because it's already detached by the
>>        time this happens
>
> Setting up a log stream is, again, something that needs to be done
> prior to starting the daemon; and the log stream's file descriptor can
> be preserved with ‘files_preserve’.

Or just set it as DaemonContext().stdout or .stderr.


>>     c) If I use the wrong pid class […] yet nothing is happening from
>>        the daemon that should be happening.
>>     d) if the daemon is already running there is no notice of this […]
>
> More specific information about these would be helpful. Set up logging,
> preserve the log through the ‘files_preserve’ option, diagnose this in
> more detail, and start a new thread to discuss the issue.

Will do.


>> I'm happy to help with development, I just need to figure out
>> bazaar. :)
>
> “Bzr Init” <URL:http://bzrinit.com/> is a user tutorial on Bazaar. The
> Bazaar documentation also has a “Centralized Workflow Tutorial”
> <URL:http://doc.bazaar.canonical.com/bzr.2.6/en/tutorials/centralized_workflow.html>
> which describes the workflow we're using.

Cool, thanks.

--
~Ethan~