[Soc-coordination] Weekly report (2nd week) - Debian GNU/Hurd Debianishinitialization

Fri Jun 28 18:54:34 UTC 2013

.. link: 
.. description: 
.. tags: gsoc, debian, hurd
.. date: 2013/06/28 17:42:08
.. title: Getting into the mood - my second week
.. slug: getting-into-the-mood-my-second-week

tl; dr version: `shutdown` works, Debian GNU/Hurd booting using
sysvinit::

    Loading GNU Mach ...
    Loading the Hurd ...
    GNU Mach 1.3.99-486
    AT386 boot: physical memory map from 0x0 to 0x9f400
    AT386 boot: physical memory map from 0x100000 to 0x1fffe000
    AT386 boot: physical memory from 0x0 to 0x1fffe000
    Enabling FXSR
    pcibios_init : BIOS32 Service Directory structure at 0xfcff0
    pcibios_init : BIOS32 Service Directory entry at 0xfc7ba
    pcibios_init : PCI BIOS revision 2.10 entry at 0xfc78c
    Probing PCI hardware.
    ide: Intel 82371 PIIX3 (dual FIFO) DMA Bus Mastering IDE
        Controller on PCI bus 0 function 9
    ide: BM-DMA feature is not enabled (BIOS), enabling
        ide0: BM-DMA at 0xc100-0xc107
        ide1: BM-DMA at 0xc108-0xc10f
    hd0: got CHS=762/128/63 CTL=c8 from BIOS
    hd0: QEMU HARDDISK, 3001MB w/256kB Cache, CHS=762/128/63, DMA
    hd2: QEMU DVD-ROM, ATAPI CDROM drive
    ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
    ide1 at 0x170-0x177,0x376 on irq 15
    Floppy drive(s): fd0 is 1.44M
    FDC 0 is a S82078B
    probing scsi 5/22: Adapteco1542FInvalidiaddresseforishpnt.with 1542.
    Invalid address for shpnt with 1542.
    probing scsi 18/22:AWesternxDigitalAWD-7000nFailed9initialization of WD-7000 SCSI card!
    probing eata on/1f0EATA0:Daddress 0x1f04in3use, skipping probe.
    probing eata on 170EATA0: address 0x170 in use, skipping probe.
    probing scsi 21/22: Iomega7parport ZIP drive ppa: Version 1.42
    ppa: Probing port 03bc
    ppa: Probing port 0378
    ppa:	 SPP port present
    ppa:	 PS/2 bidirectional port present
    ppa: Probing port 0278

    done
    scsi : 0 hosts.
    scsi : detected total.
    Partition check (DOS partitions):
     hd0: hd0s1 hd0s2 < hd0s5 >
    lpr0: at atbus0, port = 378, spl = 6, pic = 7.
    2omultibootxmodulesd/execl$(exec-task=task-create)ne=${kernel-command-line} --host-priv-port=${host-port} --device-master-port=${device-port} --exec-server-task=${exec-task} -T typed ${root} $(task-create) $(task-resume)	       task loaded: ext2fs --readonly --multiboot-command-line=root=device:hd0s1 console=com0 --host-priv-port=1 --device-master-port=2 --exec-server-task=3 -T typed device:hd0s1
    task loaded: exec /hurd/exec

    start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec init proc auth
    INIT: version 2.88 booting
    Using makefile-style concurrent boot in runlevel S.
    Activating swap...done.
    Checking root file system...fsck from util-linux 2.20.1
    hd2 : tray open or drive not ready
    hd2 : tray open or drive not ready
    hd2 : tray open or drive not ready
    hd2 : tray open or drive not ready
    end_request: I/O error, dev 02:00, sector 0
    ext2fs_check_if_mount: Can't check if filesystem is mounted due to missing mtab file while determining whether /dev/hd0s1 is mounted.
    /dev/hd0s1: clean, 43930/181056 files, 281248/723200 blocks
    done.
    Cleaning up temporary files... /tmp.
    /etc/rcS.d/S06mtab.sh: 48: /etc/rcS.d/S06mtab.sh: cannot open /proc/mounts: No such file
    Activating lvm and md swap...(default pager): Already paging to partition hd0s5!
    done.
    Checking file systems...fsck from util-linux 2.20.1
    hd2 : tray open or drive not ready
    hd2 : tray open or drive not ready
    end_request: I/O error, dev 02:00, sector 0
    done.
    Mounting local filesystems...mount: invalid option -- 'O'
    Try `mount --help' or `mount --usage' for more information.
    failed.
    Activating swapfile swap...(default pager): Already paging to partition hd0s5!
    done.
    df: Warning: cannot read table of mounted file systems: No such file or directory
    Cleaning up temporary files....
    Configuring network interfaces...inetutils-ifconfig: invalid arguments
    ifup: failed to open pid file /run/network/ifup-/dev/eth0.pid: No such file or directory
    Internet Systems Consortium DHCP Client 4.2.2
    Copyright 2004-2011 Internet Systems Consortium.
    All rights reserved.
    For info, please visit https://www.isc.org/software/dhcp/

    can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
    Listening on Socket//dev/eth0
    Sending on   Socket//dev/eth0
    DHCPDISCOVER on /dev/eth0 to 255.255.255.255 port 67 interval 6
    DHCPREQUEST on /dev/eth0 to 255.255.255.255 port 67
    DHCPOFFER from 10.0.2.2
    DHCPACK from 10.0.2.2
    can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
    bound to 10.0.2.15 -- renewal in 38544 seconds.
    done.
    Cleaning up temporary files....
    Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
    INIT: Entering runlevel: 2
    Using makefile-style concurrent boot in runlevel 2.
    Starting enhanced syslogd: rsyslogd.
    Starting deferred execution scheduler: atd.
    Starting periodic command scheduler: cron.
    Starting system message bus: dbusFailed to set socket option"/var/run/dbus/system_bus_socket": Protocol not available.
    Starting OpenBSD Secure Shell server: sshd.

    GNU 0.3 (debian) (console)

    login:

This is the result of:

* A patch by Pino Toscano that adds the `-e` and `--ifexists` flags to
  Hurds `swapon`:
  http://lists.gnu.org/archive/html/bug-hurd/2012-01/msg00060.html

* Pino Toscanos `runsystem.sysv` that starts `sysvinits` `init` (see
  below).

* A patch series to allow setting options of `procfs` translators at
  runtime (the hurdish equivalent of "remounting"):
  http://lists.gnu.org/archive/html/bug-hurd/2013-06/msg00119.html

* A trivial patch to `tmpfs`, also for updating the options at
  runtime:
  http://lists.gnu.org/archive/html/bug-hurd/2013-06/msg00126.html

* A patch series adding `--fake` and `--no-mtab` to our `mount`
  utility, fixing `-oremount` with one parameter (e. g. `mount
  -oremount,ro /`) and finally fixing a subtle bug in an internal
  look-up function:
  http://lists.gnu.org/archive/html/bug-hurd/2013-06/msg00135.html

* A minor tweak to `checkroot.sh`.

* A minor tweak to `mountall.sh`. Turns out that there are two init
  processes at this point, `pidof -s` takes care of that. Not sure why
  this does not happen on Linux, maybe it even does? Or it is related
  to us using `inits` undocumented `-i` flag to make it ignore the
  fact that it isn't pid 1. Needs further investigation.

* Disabling `killprocs` and `sendsigs`. See below.

* A minor tweak to `mount-functions.sh`. Same as last week though.

* Using the same `inittab` as Debian/Linux uses (we seem to ship our
  own, though I must admit I'm not sure it comes from, `dpkg` doesn't
  know about this file on my installations).

* Using the `reboot` and `halt` utilities from the `sysvinit` package
  instead of our own.

This yields a surprisingly functional Debian/Hurd system with
`sysvinit`. There are still lot's of loose ends, I populated my
`status page </gsoc>`_ with them.

I spent quite some time in #hurd on freenode and Pino Toscano offered
to send me some old notes of his about the sysvinit issue. This turned
out to be most helpful as his notes were about plugging `sysvinits`
init into the boot process. This was the next logical thing to do for
me, so first thing Monday morning I just put his `runsystem.sysv` file
with some very minor tweaks (`my version
</gsoc/heap/runsystem.sysv>`_) into my overlay and had a working system
in front of my eyes. That certainly was a nice way to start a week of
work :), so many thanks again Pino!

For your convenience I've created an `overlay
<gsoc/heap/second-week-overlay.tar.xz>`_ with all necessary changes
and recompiled binaries. Be careful though, you might want to backup
`/sbin/reboot` first or you won't be able to reboot your system! Of
course my tool does that for you, and I've managed to create a
reasonably small (253 mb) `image </gsoc/heap/image.qcow2.xz>`_ thanks
to the `zerofree` tool, the process of compacting images is
implemented in `hurdtest` as the `compact` subcommand. I also added
`sysvinit` specific tests. So if you want to try `hurdtest`, download
the image and unxz it, my tool expects the image to be in the current
working directory and be named `image.qcow2`. Download and untar an
overlay and execute::

  % hurdtest -logfile log -sysvinit overlay path/to/second-week-overlay

If you specify `-logfile` you can execute `tail -f path/to/log` to see
what the VM is printing to the console in real time. The `sysvinit`
specific tests demonstrate that indeed `sysvinit` is managing the
system and that e. g. `init 0` or `shutdown -r now` works.

There are at least three major problems that I will need to tackle for
this project:

* `/bin/init` is not running as pid 1.

* `kill(-1, SIGTERM)` kills essential processes.

* There is no `/proc/mounts`.

I think the last issue is what I'm going to look at next week. There's
information about this in the Hurd wiki and I've spoken with Richard
Braun about it and he drilled me to think about the hurdish way to
solve this. I think I've got a pretty good picture of how to implement
this in my head and am eager to give it a go. This will involve adding
a function to Hurds filesystem interface that allows one to query an
active translator about his mount options and "source" and about all
active and passive translators that are attached to one of its nodes.

This is where the fun actually begins. This is changing an integral
part of an operating system, but this is okay since it is just another
program written in `c`, operating in userspace. But hey I know that,
that's just `c` and if something goes wrong (and it always does) I
fire up gdb and have a look. And mind you, I'm running all this as an
unprivileged user, no need to jeopardize the entire system for that.

Besides implementing `/proc/mounts` I will also add `-O`,
`--test-opts` to `mount`, and `-v`, `--verbose` to `swapo{n,ff}`.

PS: My blog is still not properly registered to planet.d.o. I asked
planet@ about that, but got no response. Any help? I never imagined
blogging being so stressful, I care about losing potential readers :/