[Pkg-xen-devel] Bug#776450: Xen PVH support for grub-xen in Buster

[Colin Watson]

On Sat, Jan 05, 2019 at 10:51:13PM +0100, Hans van Kranenburg wrote:
> On 1/5/19 8:04 PM, Colin Watson wrote:
> > I think I'm OK with cherry-picking the relevant patch stack.  Presumably
> > this would need to be in new grub-xen-pvh-bin / grub-xen-pvh binary
> > packages, as is usual for separate platform builds?
> 
> I was thinking about having an additional...
> 
>     /usr/lib/grub-xen/grub-i386-xen_pvh.bin
> 
> ...in the grub-xen-host package, built with the same config file
> (debian/grub-xen-host_grub.cfg in the packaging) and then...
> 
>     /usr/lib/grub/i386-xen_pvh/*.mod
> 
> ...in grub-xen-bin, just as an additional third option besides i386-xen
> and x86_64-xen? This would be the most convenient "upgrade path". If you
> want to switch to PVH, just change kernel line in the xen guest config
> file, and set type='pvh'.

Ah yes, I'd forgotten that i386-xen and x86_64-xen were both in
grub-xen-bin.  I agree it seems reasonable enough to combine them, which
simplifies things, and given that the boot loader (at least the first
stage - see below) is read from the dom0 filesystem, it makes sense to
include an image in grub-xen-host, although we may or may not be able to
use the exact same config file.

> > Can you elaborate on what you mean by "use as kernel image for the Xen
> > PVH virtual machine"?  In other words, what does the process of
> > installing a boot loader in a PVH guest look like?
> 
> You don't. There is no BIOS, no boot loader, it is like booting a PV
> guest with grub. The whole 'trick' about PVH is that it's HVM, but then
> without the qemu etc part.
> 
> https://wiki.xen.org/wiki/Virtualization_Spectrum#Almost_fully_PV:_PVH_mode
> 
> So, if I have this grub.cfg... (as example, not what we want for the
> package)
[...]

Thanks for clarifying all that.

> So this is what I'm used to [0], but the way in which the Debian
> grub-xen packages work is a bit different of course, since it tries to
> do most of the process inside the virtual machine, and has a very
> generic config file that delegates as much as possible to additional
> grub config with access to modules inside the virtual machine itself.
[...]
> > (Compare
> > https://salsa.debian.org/grub-team/grub/blob/master/debian/patches/grub-install-pvxen-paths.patch
> > which we carry for PV guests, although I gather PVH is sufficiently
> > different that the same approach probably doesn't work.)
> 
> Hm, I have never seen this /boot/xen/pvboot-*.elf stuff before, and I
> also don't see any of those files in the grub-xen-* packages?

Anything in /boot is copied/generated by grub-install rather than being
shipped in the packages themselves.

Let me give some background here.  The Xen PV boot protocol was the
result of a series of discussions between me, Ian Jackson (Xen), Ian
Campbell (Xen), Vladimir Serbinenko (GRUB), and possibly a few others
I've forgotten, and I don't recall to what extent we ever wrote down the
entire rationale in one place.  In fact
https://wiki.xen.org/wiki/PvGrub2 seems to have most of it now that I
look, but I'll try to summarise it all here.

Historically (before GRUB 2.02), there were two main choices for booting
a PV guest using GRUB:

 * Use pygrub.  This was very flexible, because it parsed boot loader
   configuration files from the guest's filesystem and decided what
   kernel and initramfs to boot based on that.  However, getting the
   configuration file parsing right was a difficult and moving target,
   and this system was inherently somewhat insecure because it required
   complex parsing of untrusted guest inputs in the dom0.

 * Use the PV port of GRUB Legacy (pvgrub).  This generally worked well
   and fixed the inherent security problems of pygrub because all the
   parsing work was done in the guest context, but it was a very old
   fork of unmaintained code and expected the GRUB Legacy configuration
   file syntax.  Guests running on a host using this generally had to
   carry around otherwise-obsolete code to generate appropriate menu.lst
   files so that they could declare how they expected to be booted.

Of course it was also possible to use pvgrub with the sort of scheme you
describe, where the host declares how the guest kernel should boot.
This works well enough in a small environment where there's a
substantial amount of cooperation between the people running the host
and the people running the guests, or where all the guests are
sufficiently similar.  But it's not great for a larger or more generic
environment, and we didn't want to rely on that degree of cooperation.
(However, if you're operating a Xen host and that approach works for
you, by all means go for it!  I understand that there exist environments
where that approach is more convenient.)

So, around the time Vladimir was doing the core PV porting work for GRUB
2, we were working on a better design for this.  The first pass was just
to provide the host with a pvgrub2 image which it would boot in the
guest context, configured to look for /boot/grub/grub.cfg in the guest.
As long as the guest's needs are simple enough, this works OK.  However,
it relies on certain compatibility assumptions: specifically that the
guest doesn't use any configuration file features from versions of GRUB
more recent than the version that the host has, and that the guest's
grub.cfg doesn't need to load any modules not present in the host's GRUB
image (since modules in the guest's filesystem may very well not be
compatible with the host's image).

So, the second pass was the two-stage arrangement that you see in the
current packaging.  The host has a fairly simple generic PV GRUB image
which it boots in the guest context; the first thing that that image
does is look for a second-stage loader in a defined place in the guest
filesystem and chainload it (with fallbacks in case it isn't present).
This means that the guest can take full control of its own booting
process: it can use any features of whatever version of GRUB it likes,
as long as it supports the correct entry point, and needs no changes to
the host in order to do so.  This is particularly good if you're working
on GRUB itself, of course, but the looser coupling also generally makes
the process more robust.

Given what you've described, I see no intrinsic reason we couldn't do
the same thing with PVH, but I'm pretty sure that GRUB's Xen multiboot
support doesn't yet support using the PVH entry point, so I don't think
it will work with the code as it stands.  It probably makes more sense
to package what we have today (which I can do) and then work on the
two-stage system as a further refinement.

-- 
Colin Watson                                       [cjwatson at debian.org]