Bug#799480: grub-xen-host: XEN domU crash when PV grub chainloads 32-bit domU grub

[Ian Campbell]

On Sun, 2015-09-20 at 20:15 +0200, Andreas Sundstrom wrote:
> On 2015-09-20 18:51, Ian Campbell wrote:
> > On Sat, 2015-09-19 at 18:49 +0200, Andreas Sundstrom wrote:
> > > Package: grub-xen-host
> > > Version: 2.02~beta2-22
> > > Severity: important
> > > 
> > > Dear Maintainer,
> > > 
> > > Using 64-bit dom0 and 32-bit domU PV (para-virtualized) grub
> > > sometimes
> > > fail when chainloading the domU's grub. 64-bit domU seem to work 100%
> > > of the time.
> > Which grub are you starting with from dom0?
> > 
> > If you want to boot a 32-bit guest (which includes chainloading a 32
> > -bit grub) then you must start with the 32-bit grub-i386-xen.bin grub
> > binary to create a 32-bit guest.
> > 
> > kexecing from 64-bit to 32-bit is not possible in the general case. In
> > fact I thought it was _impossible_ in all cases and would have ruled it
> > out as something you might be doing, except some of these registers
> > look like 64-bit values:
> 
> As you say it has not been possible at any time to use 64-bit grub from
> dom0 and then load either 32-bit grub or linux kernel from domU.
> 
> I am using /usr/lib/grub-xen/grub-i386-xen.bin when I start my i386
> domU's

OK good (well, bad, because now I have no idea what is going wrong...)

> Thanks for your great blog entry about this by the way:
> https://blog.xenproject.org/2015/01/07/using-grub-2-as-a-bootloader-for-x
> en-pv-guests/
> I have used it to get a better understanding of the whole process.
> > > (XEN) rax: 0000000000000000   rbx: 0000000000000000   rcx:
> > 0000000000000000
> > > (XEN) rdx: 0000000000000000   rsi: 0000000000499000   rdi:
> > > 0000000000800000
> > > (XEN) rbp: 000000000000000a   rsp: 00000000005a5ff0   r8: 
> > >  0000000000000000
> > > (XEN) r9:  0000000000000000   r10: ffff83023e9b9000   r11:
> > > ffff83023e9b9000
> > > (XEN) r12: 0000033f3d335bfb   r13: ffff82d080300800   r14:
> > > ffff82d0802ea940
> > > (XEN) r15: ffff83005e819000   cr0: 000000008005003b   cr4:
> > > 00000000000506f0
> > > (XEN) cr3: 0000000200b7a000   cr2: 0000000000000000
> Well I don't know but I guess the XEN hypervisor is always running in
> 64-bit mode yes?
> I suppose that maybe even if the domU is 32-bit any errors showing up in
> "xl dmesg"
> reflects the mode that the hypervisor is run in?

I think it's supposed to reflect the mode which the processor is in at the
time. I trimmed the quotes but there was a line in the dump which said:

(XEN) RFLAGS: 0000000000000246   EM: 1   CONTEXT: pv guest

Suggesting that this was guest context (this string doesn't distinguish 32-
from 64-bit).

Actually, I just spotted:
(XEN) domain_crash_sync called from entry.S: fault at ffff82d08021feb0 compat_create_bounce_frame+0xc6/0xde

where compat == 32-bit, so that bit is correct.

So I think the large register values are a red-herring.

I think it would be worth reporting this to upstream (both Xen and Grub),
would you mind doing so?

Ian.