Need help debugging shutdown issues

[Nikolaus Rath]

Michael Biebl <biebl-8fiUuRrzOP0dnm+yROfE0A at public.gmane.org> writes:
> Am 06.11.2014 um 07:39 schrieb Nikolaus Rath:
>> Nikolaus Rath <Nikolaus-BTH8mxji4b0-XMD5yJDbdMReXY1tMh2IBg at public.gmane.org> writes:
>>> On 11/05/2014 02:51 AM, Michael Biebl wrote:
>>>> Am 05.11.2014 um 03:35 schrieb Nikolaus Rath:
>>>>> Hello,
>>>>>
>>>>> I'm using testing with systemd as pid 1, and my system does not shut
>>>>> down (or reboot) properly.
>>>>>
>>>>> That's a lot better than not booting properly, but I'd nevertheless like
>>>>> to fix it :-).
>>>>>
>>>>> I tried following the instructions at
>>>>> http://freedesktop.org/wiki/Software/systemd/Debugging/#index2h1 and
>>>>> found that
>>>>>
>>>>> sync && reboot -f
>>>>> sync && poweroff -f
>>>>>
>>>>> work just fine. When rebooting or powering off with systemctl, however,
>>>>> the system is still active after 10 minutes of waiting.
>>>>>
>>>>>
>>>>> I then proceeded to add
>>>>>
>>>>> systemd.log_level=debug systemd.log_target=kmsg log_buf_len=1M
>>>>> enforcing=0
>>>>>
>>>>> to the kernel command line and created the
>>>>> /usr/lib/systemd/system-shutdown/debug.sh file. I noticed that the
>>>>> system-shutdown directory did not exist - does that mean this method is
>>>>> not applicable in Debian?
>>>>>
>>>>> In any case, I did not get the /shutdown-log.txt file after trying to
>>>>> shutdown, so I activated the debug shell instead.
>>>>>
>>>>> On the debug shell, I ran
>>>>>
>>>>> systemctl reboot &
>>>>> sleep 120
>>>>> dmesg > dmesg.txt
>>>>> systemctl status > systemctl-status.txt
>>>>> systemctl list-units > systemctl-units.txt
>>>>> systemctl list-jobs > systemctl-jobs.txt
>>>>>
>>>>> This gave me a huge amount of information (included below), but I don't
>>>>> really know what to do with it. For example, is it a problem if
>>>>> samba.service has status "stop running"? Or is the problem with
>>>>
>>>> This is most likely a samba bug and a duplicate of [1].
>>>> I assume, you're not running samba 2:4.1.13+dfsg-2, which is supposed to
>>>> fix that issue.
>>>> Could you upgrade to that version and test again?
>>>
>>> I'm at 2:4.1.13+dfsg-2 now, problem is still the same unfortunately.
>> 
>> 
>> Ok, this probably sounds weird, but I confirmed it several times:
>> 
>> - when I run "systemctl reboot/poweroff" on a console, it works
>> - when I run "systemctl reboot/poweroff" on an X11 terminal, it
>>   hangs
>> 
>> Additionally, when running "systemctl reboot" from X11 the system
>> becomes unusable after a fraction of a second - the screen goes blank
>> and I can't switch to a text console anymore either. I have tried to
>> issue the command and then quickly switch, but this did not work either.
>> 
>> When I'm running "systemctl reboot" from a console, the system stays
>> usable. I can see the status messages, switch to other consoles and use
>> the debug shell. But this doesn't help much, because the system shuts
>> down just fine.
>> 
>> 
>> I have tried to start the following script in the debug shell *prior* to
>> executing "systemctl reboot" in X11:
>> 
>> #!/bin/sh
>> exec > shutdown.log
>> while true; do
>>   echo
>>   echo
>>   date
>>   systemctl list-jobs
>>   systemctl status
>>   sync
>>   sleep 10
>> done
>> 
>> The last entry in shutdown.log is then:
>> 
>> Wed Nov  5 22:28:29 PST 2014
>> JOB UNIT                                                           TYPE  STATE  
>> 775 reboot.target                                                  start waiting
>> 776 systemd-reboot.service                                         start waiting
>> 777 shutdown.target                                                start waiting
>> 782 local-fs.target                                                stop  waiting
>> 783 systemd-remount-fs.service                                     stop  waiting
>> 838 networking.service                                             stop  running
>
>>              |-networking.service
>>              | `-control
>>              |   |-3416 /bin/sh -e /etc/init.d/networking stop
>>              |   |-3427 ifdown -a --exclude=lo
>>              |   |-3463 /bin/sh -c dhclient -v -r -pf /run/dhclient.br0.pid -lf /var/lib/dhcp/dhclient.br0.leases br0
>>              |   `-3464 dhclient -v -r -pf /run/dhclient.br0.pid -lf /var/lib/dhcp/dhclient.br0.leases br0
>
>
> Try running "systemctl stop networking.service" before you
> shutdown/reboot. This service seems to be hanging in "ifdown -a".
> Might be dhclient or ifdown.d hook script causing a dead lock.

I tried that, but it still hangs. That said, why would you expect this
to be a problem only when rebooting from X11?

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«