[Debian-med-packaging] Help with systemd to start shiny-server needed

[Christian Seiler]

On 03/24/2017 04:21 PM, Andreas Tille wrote:
> I intend to package shiny-server and have prepared some preliminary
> packaging in Debian Med Git[1].  When trying to install the resulting
> package I get: [...]
> 
> Mar 24 16:20:23 sputnik systemd[1]: Starting ShinyServer...
> Mar 24 16:20:26 sputnik systemd[1]: shiny-server.service: PID file /var/run/shiny-server.pid not readable (yet?) after start-post: No such file or directory
> Mar 24 16:20:31 sputnik systemd[1]: Failed to start ShinyServer.

Well, your service doesn't appear to write out a PID file
(or delete it before it exits again), hence systemd can't
find that PID file and will consider the service to be
failed.

Your logs indicate that the process seems to have exited,
but whether that's because of SIGTERM sent to it or not
is unclear at this point.

First thing is you need to figure out why your service
doesn't appear to write a PID file at all. From the logs
you posted, it could also be that the service exits
prematurely.

Why are you using Type=simple + PIDFile= anyway?

sleep 3 / 5 in ExecStartPost=/ExecStopPost= also seems to
be completely wrong to me.



>From my perspective, when writing a systemd unit file for a
service, I'd like to follow the following guidelines:


 - Ideally the service supports systemd's notification
   protocol, in which case I'd use Type=notify and no PID
   file. Doesn't appear to be the case here.

 - If the service is just a program you start and it does
   not fork (so running it on the command line will have
   it be active until you press Ctrl+C or similar), then
   the best thing one can do is Type=simple, but no PID
   file, the process started by systemd is considered to be
   the main PID file of that service.

   This is not 100% ideal, as there will be no notification
   of whether the service has been started or not (systemd
   will just assume the service is up if fork+exec works),
   but without a notification protocol, that's the best
   one can do.

 - If a service forks, then I'd use Type=forking. If the
   service supports writing out a PID file [1], then I'd
   tell the service to write a PID file in /run [2], and
   tell systemd to look for it (via PIDFile=).



I don't know much about your specific code, so you'd have to
test this on the command line first (to see how the process
reacts), but you'd either just want a Type=simple OR you'd
want Type=forking with PIDFile=... But you wouldn't want to
have Type=simple with PIDFile=, that just doesn't make any
sense.

And the sleep stuff in *StartPost seems completely wrong to
me. Especially ExecStopPost: please use KillMode=control-group
or KillMode=mixed instead (see docs) if you want to ensure
that all processes of a service have exited.



Regards,
Christian


[1] Most services actually get PID file handling and forking wrong,
    which is why many init system developers have developed their own
    startup notification mechanisms for services that are way easier
    to get right.

    What should happen for forking services is (in this order):

     - process forks (possibly twice)
 
     - parent process stays alive and waits for child process to
       signal it that initialization is complete (there are various
       ways of doing this, easiest is just using a pipe(2) and
       closing it from the child when that's done)

     - child process initializes (opens all sockets, log files,
       etc.)

 -   - child process signals parent process that it's done

     - parent process writes out the PID file (with the PID of the
       child) that the child is now up and running

     - parent process exits to signal the caller that the daemon
       has now been initialized successfully

   (Alternatively you can already initialize in the parent process
   and fork only afterwards when you know you're all set.)

   Basically, from the outside, the proper interface should be:

     - process that was started exits with code 0 and a PID file
       has been written at that time, plus the PID in the PID file
       exists: service is considered started successfully

     - process that was started exits with non-zero code: failure

     - process that was started exits with code 0 but PID file
       does not exist: shouldn't happen

   I see way too many programs out there that don't get this
   completely right. Common mistakes:

     - PID file is written in child process, but possibly at a
       point after parent process has exited

     - PID file is written unconditionally after fork, even if
       initialization in child process has failed

     - parent process exits before child is properly initialized

[2] Btw. in Debian you should use /run instead of /var/run, which
    nowadays is just a symlink to /run anyway. For early-boot
    services this is required (because /var could be a remote file
    system), but even if it's not required, unless there's a fixed
    value set by upstream that you don't want to patch out for no
    good reason, please use /run directly anyway.