[Pkg-xen-devel] Bug#703586: Bug#703586: Xen fails to boot Linux dom0 under UEFI

[John Keates]

I’ll explain what’s happening first, and list the steps after that.

First: pep is short for the Xen target x86_64-pep. It’s a target you can enable when configuring Xen to create an EFI binary.

What is happening with Xen on UEFI via Grub is that it doesn’t give the kernel any info on the ACPI root pointer.
Basically, this means that Linux won’t be able to do ACPI, and therefore a ton of hardware functionality is lost.
The reason why this used to happen was the fact that Xen got a e-801 map from Grub for some reason.
Not sure if that exact issue is still the source of this current issue, but Xen via Grub on EFI simply doesn’t work.

But when you use Xen on UEFI via the EFI binary (xen.efi) it works just fine, the kernel gets to know about RSDP so it can do it’s ACPI stuff.

Loading xen.efi can be done in different ways. The fastest way is to let efibootmgr know about it and booting it directly.
Alternatively, you can simply test it by manually loading it from the UEFI shell, and as a third option, you can EFI chain load it from Grub.
The chain loading pretty much means that grubx64.efi loads xen.efi just as you would from the shell or the bootmgr.

There is one small catch: Xen on native EFI doesn’t give you a normal efivars system in dom0.
This has to be fixed on a lower level in either Linux or Xen. From what I’ve found out so far, the efivars are passed in a different way to Linux,
but Linux doesn’t know about it and therefore cannot use it. This is the only downside to the whole thing. However, if you only want to boot using the EFI boot menu, this is not really a problem: you can still choose to boot xen.efi or grubx64.efi and when you boot grub and a non-Xen entry, you can use efivars (and efibootmgr) all you want.

Steps:

1. Rebuild the debian package with a small change

Do your usual apt-sourcing and build-depping, but add the pep target to debian/rules:

(I put it right underneath include debian/rules.defs)

DEB_CONFIGURE_EXTRA_FLAGS  += --enable-targets=x86_64-pep

then do your usual binary run to produce the files needed.
You’ll find xen.efi in ./debian/build/build-hypervisor_amd64_amd64/xen/xen.efi

2. Allow UEFI to find your xen.efi

UEFI uses your ESP to launch the efi binaries, so that’s where it needs to go.
Simply put (not symlink) xen.efi in /boot/efi/EFI/debian/

3. Configure dom0 booting

In order for Xen to boot your dom0, you’ll have to do two things:

- Get the kernel and initrd on the ESP as well
- Configure Xen to load a specific kernel and initrd

The kernel and initrd are easy, just copy them from /boot/ to /boot/efi/EFI/debian/
I used my most current kernel and initrd.

Xen.efi configuration is easy as well, since you only need to create a short ini-style cfg file.
This configuration file needs to be on your ESP as well, with the kernel, initrd and xen.efi files.
All in the same directory!

Xen.cfg is fairly straightforward, I’ll just put mine here:

[global]
default=debian8

[debian8]
options=console=vga,com1 com1=115200 loglvl=all noreboot
kernel=vmlinuz-3.14-2-amd64 ignore_loglevel  root=/dev/mapper/clava-root ro quiet #earlyprintk=xen
ramdisk=initrd.img-3.14-2-amd64

And that’s it! Xen.efi now boots super-fast, dom0 gets booted really fast as well, everything is working!

Now, instead of messing around in the UEFI setup ROM to create a boot entry, you might as well do it from your non-xen system while it’s still booted.
Simply add an entry with efibootmgr:

sudo efibootmgr -c -g -d /dev/sda -p 1 -w -L "Xen EFI Loader" -l '\EFI\debian\xen.efi'

Where /dev/sda is the disk with the partition containing your ESP and -p 1 is set to the partition number for that ESP partition (in my cast it’s partition 1).

You can make the Xen entry the default or just manually boot it whenever you want.
Buy the way, if you run a grub-update right now, it’ll find your efibootmgr entry and add it to the list for chainloading.

The end.


Well, almost, because those manual steps are pretty easy to incorporate in the related packages.

First: xen-4.4.0 simply needs to enable the target x86_64-pep by default. Without it no Xen EFI.

Second: The package that creates the grub.d xen_linux snippet might be extended to move xen.efi, vmlinuz, initrd and a xen.cfg to the ESP

Kernel commandline options can be taken from /proc/cmdline and/or grub’s /etc/default/grub and added to the xen.cfg automatically.
The efibootmgr entry can be added automatically as well.

I don’t have a patch handy right here, I use a hackish bash script for now to produce the desired results for any servers I set up that have UEFI without CSM but need to run Xen. A patch would be pretty neat of course. I don’t really know how to make official patches for debian packages, but if nobody wants to do it, I can give it a go, manuals and tutorials everywhere anyway.

John

On 07 Sep 2014, at 08:12, Ian Campbell <ijc at hellion.org.uk> wrote:

> On Sun, 2014-09-07 at 00:02 +0200, John Keates wrote:
>> How do I assist with getting this in for Jessie?
>> 
>> I have this working in a fairly easy setup, it basically only requires
>> the pep target to be on for debian’s Xen package,
> 
> What is "pep"?
> 
>> and a tiny bit of infrastructure to get xen.efi, vmlinuz, an initrd
>> and a xen.cfg on to the ESP partition and letting the efibootmgr know
>> about it.
>> 
>> I already have a oneliner to do this manually, but it isn’t that hard
>> to automate, unless that tiny bit of infrastructure regarding the
>> kernel bits and efi boot is creating a ripple touching many other
>> packages, or if there are rules I am not aware of that make this
>> really hard to make conform to those rules.
> 
> Probably the best approach would be to outline the step(s) here first
> and then we discuss how to take it from there.
> 
> If you happen to already have a patch against the packaging itself then
> you also might as well post it for reference even if you think it isn't
> going to be the preferred solution.
> 
> Ian.
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4124 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-xen-devel/attachments/20140907/531e6c4d/attachment-0001.bin>