Bug#808151: systemd: failed to start remount root and kernel file system

[Frank B. Brokken]

Dear Michael Biebl, you wrote:

> > This information is available at https://www.icce.rug.nl/systemd in the files 
> > initramfs.debug and alb.
> 
> Hm, unfortunately the journal dump is incomplete again. I have no idea why

Remarkable. I made it available the way I got it, so that's apparently what
there is.

> > booting procedure. You're sure it can't be some timing problem? 
> 
> Well, what kind of timing problem do you have in mind?

Don't know: I didn't design systemd. But if it's doing things in parallel then
maybe on newer, faster, computers things might have completed, like remounting
/usr rw before it's actually used. A race condition might then explain why the
problem doesn't always show itself, and why chances of failure are reduced
when more time is spent writing debug/verbose messages.

 
> So far, the only thing I can say for sure looking at the initramfs log,
> is that /usr has been mounted successfully in the initramfs.
> 
> "Something" apparently causes /usr to be unmounted later on. Which part
> and why that is, is not clear yet.
> 
> Do you have any (custom) init scripts in /etc/rcS.d/ which fiddle around
> with mount settings, run telinit or stuff like that?

Nope.

> I'm running out of ideas, tbh.

Well, that's already a *lot* more than I could offer myself :-) But
fortunately (for me, but hard to fix, I realize), the problem doesn't emerge
all the time. If rebooting every now and then gets me a running system, then
so be it. The FailureAction=reboot-force entry in systemd-remount-fs.service
already has proven to be my friend :-)


> If you suspect the remount service to be the cause for this, let's
> output the mounts before and after
> For that run
> $ systemctl edit systemd-remount-fs.service

When I issue that command I get the reply

Warning: systemd-remount-fs.service changed on disk. Run 'systemctl
daemon-reload' to reload units.

I guess the warning is obvious as I edited the file 

    /lib/systemd/system/local-fs.target.wants/systemd-remount-fs.service

to prevent the reboot action. So I did as advised and reran the command,
but got an empty file in my editor, while the following message was shown
after ending the editor:

Editing "/etc/systemd/system/systemd-remount-fs.service.d/override.conf"
canceled: temporary file is empty.

> Then add
> [Service]
> ExecStartPre=/bin/sh -c 'echo "before rootfs remount"; findmnt'
> ExecStartPost=/bin/sh -c 'echo "after rootfs remount"; findmnt'
> 
> Reboot and attach the journal log again.

Instead of running the systemctl command I edited the file
/lib/systemd/system/local-fs.target.wants/systemd-remount-fs.service and added
the lines you suggested. My next e-mail is about the contents of journal log.

Thereafter I'll try to downgrade to the previous version to see what
happens then.


-- 
    Frank B. Brokken
    Center for Information Technology, University of Groningen
    (+31) 50 363 9281 
    Public PGP key: http://pgp.surfnet.nl
    Key Fingerprint: DF32 13DE B156 7732 E65E  3B4D 7DB2 A8BE EAE4 D8AA
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-systemd-maintainers/attachments/20151219/3e7dbbc6/attachment-0002.sig>