[Pkg-xen-devel] Bug#988333: linux-image-5.10.0-6-amd64: VGA Intel IGD Passthrough to Debian Xen HVM DomUs not working, but Windows Xen HVMs do work

[Chuck Zmudzinski]

On 10/23/2021 11:11 AM, Hans van Kranenburg wrote:
> Hi!
>
> On 10/19/21 5:44 AM, Chuck Zmudzinski wrote:
>> On 5/10/2021 1:33 PM, Chuck Zmudzinski wrote:
>>> [...] with buster and bullseye running as the Dom0, I can only get the VGA/Passthrough feature to work with Windows Xen HVMs. I would expect both Windows and Linux HVMs to work comparably well.
> You don't mention the used Xen version (Debian package version) for
> buster and bullseye anywhere, so I'll assume it's the latest
> 4.14.3-1(~deb11u1) one.

Yes, That's the version. The original report from five months ago was an 
earlier version
but the latest version still behaves the same way. I just tested it a 
couple of days
ago.

>
>> [...]
>>
>> The biggest problems were that the Dom0 reported problems
>> with IRQ 16 being disabled after starting the bullseye HVM DomU,
>> and only xl destroy could be used to stop the corrupted process.
> Well, at least we have an error somewhere already. That's a starting point.
>
> Can you share the domU config file?

Yes, here it is:

builder = 'hvm'
memory = '3072'
vcpus = '4'
device_model_version = 'qemu-xen'
# device_model_version = 'qemu-xen-traditional'
# This is now bullseye
disk = ['/dev/systems/linux,,xvda,w','/dev/data/linuxdata,,xvdb,w']
name = 'bullseye-hvm'
vif = [ 
'mac=00:16:3E:27:2C:AA,model=e1000,script=vif-route.hvm,ip=192.168.1.4' ]
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
boot = 'c'
acpi = '1'
apic = '1'
viridian = '1'
xen_platform_pci = '1'
serial = 'pty'
vga = 'none'
sdl = '0'
vnc = '0'
gfx_passthru = '1'
pci = [ '00:1b.0', '00:14.0,rdm_policy=relaxed', '00:02.0' ]

>
> And, other configs you need to have in place to exclude the devices from
> being seen as normal devices directly in dom0? (I haven't used
> passthrough myself yet, but I read that this is needed.)

I run this script in Dom0 before starting the domain:

#!/bin/bash
modprobe xen-pciback
xl pci-assignable-add 00:02.0
xl pci-assignable-add 00:14.0
xl pci-assignable-add 00:1b.0
xl pci-assignable-list

The script makes the Intel IGD, USB 3.0 controller, and
sound device available to the domain. the pci = ...
statement in the domain config corresponds to these same
three PCI devices.

>
> Can you share more verbose logging done by xl create when using xl -vvv
> create <configfile>?

I don' have time now, but will do this and report tomorrow.

>
> But, AFAIK what you want to do should be possible yes.
>
>> The bullseye HVM DomU still fails to boot on an up-to-date
>> bullseye Xen Dom0 configured to pass through the same PCI/IGD
>> devices. The bullseye HVM DomU with IGD passthrough has so
>> far only been verified to work on an old, slightly modified
>> jessie Xen Dom0.
>>
>> More Details: These latest tests are with linux version 5.10.70-1
>> for bullseye stable. For the jessie Dom0, which worked with the
>> unmodified bullseye HVM DomU, I had to add a few patches to
>> the old jessie Xen packages so the unmodified bullseye Xen HVM
> Ok, yes, clear, that makes the domU kernel not the primary suspect.
>
>> These tests demonstrate that a fix for this bug is possible in src:xen
>> rather than in src:linux, but the patches needed to fix this bug in
>> Xen 4.14, which is the version of Xen on bullseye, are not yet
>> identified.
> It might also be possible (just a wild guess) that for Xen 4.14, the
> options in the domU config file need to be different than for Xen 4.4.

They are a little different already, 4.4 did not need the rdm_policy 
setting.
But you are right, there are other settings I haven't checked yet. I 
will report
on some more tests I have done tomorrow when I have more ti

>
>> I will continue to investigate this issue and try to bisect the problem
>> as it recurs in Dom0 for some version of Xen > 4.4 and <= 4.14. It
>> will obviously take some time since there are so many differences
>> between Xen 4.4 and 4.14.
> If you can make progress on that, and find an actual commit that changes
> the behavior, then we're probably at 95% towards finding a cause and
> solution. :) That'd be great.
>
> A possible time-saver that I can recommend is to send a post to the
> upstream xen-users list [0] about this already. Like "Hi all, I'm
> starting a HVM Linux domU with Linux 5.10.70 on a Xen 4.14.3 system with
> also 5.10.70 dom0 kernel, with this and this domU config file. It fails
> to start, this is the xl -vvv create output, and this error (the irq
> stuff) appears in the dom0 kernel log.". Try to keep it simple and not
> too long initially, without the surrounding stories, to increase chance
> of it being fully read.

I can do this soon - I have some more interesting tests to share
here and with the Xen developers upstream.

>
>> If I find a fix in src:xen for Xen >=4.14 Dom0 on bullseye or sid, I will
>> reassign #988333 to src:xen myself. Until then, I will leave it to the
>> discretion of the Debian Kernel Team to decide whether or not to
>> reassign it to src:xen now.
> Yes, that makes sense indeed, I'll do it in a minute. Even while we
> don't know if it has to do with the Xen or dom0 kernel code, it's more
> likely that in either case, we'll end up asking the upstream Xen people
> about it.
>
> Have fun,

OK, I will!

Cheers,

Chuck