[Pkg-xen-devel] Bug#963607: Bug#963607: xen-hypervisor-4.11-amd64: Xen Hypervisor kernel fails to load arcmsr module with "arcmsr0: dma_alloc_coherent got error" message.

[Alex Sanderson]

Hi,

On 1/07/2020 02:05, Hans van Kranenburg wrote:
> Hi,
>
> On 6/25/20 1:44 PM, Alex Sanderson wrote:
>> Hi Hans,
>>
>> Thank you for your assistance with this.  I hesitated to log this with
>> xen-dev but thought I should wait for a response here first. 
>>
>>
>> On 25/06/2020 01:30, Hans van Kranenburg wrote:
>>> Hi Alex,
>>>
>>> On 6/24/20 12:31 PM, Alex Sanderson wrote:
>>>> Package: xen-hypervisor-4.11-amd64
>>>> Version: 4.11.3+24-g14b62ab3e5-1~deb10u1
>>>> Severity: important
>>>>
>>>> Dear Maintainer,
>>>>
>>>> After updating to Buster and Xen 4.11 our machine no longer boots the Xen kernel.  The default kernel 4.19.118-2+deb10u1 boots normally.
>>> When booting with Xen, the computer first starts the Xen hypervisor
>>> code. This is the part where you see all the lines with (XEN) at the
>>> beginning appear.
>>>
>>> Afterwards, it starts the same 4.19.118-2+deb10u1 Linux kernel that is
>>> used when running without Xen, but it's started as the first virtual
>>> machine, that has extra privileges to access all hardware.
>>>
>>> So, Linux vs. Xen + Linux.
>>>
>>>> The machine has an Areca 1882IX-16 card in it when the arcmsr module
>>>> tries to load the following error appears. 
>>>>
>>>> 	Areca RAID Controller0: Model ARC-1882, F/W V1.56 2019-07-30
>>>> 	arcmsr0: dma_alloc_coherent got error
>>>>
>>>> No drives are discovered and the initramfs prompt is shown.
>>> Ok, so booting the Xen part succeeded, but apparently, when starting the
>>> Linux kernel inside, there's apparently a problem with accessing the
>>> raid controller hardware. Interesting.
>>>
>>> This likely means it's not a problem in the Debian packaging part, it's
>>> a problem somewhere in the upstream Xen or Linux code. That means that I
>>> cannot solve this for you, but I can help with tips to gather the right
>>> information, and help finding out what the best place is where we can
>>> report the issue.
>>>
>>>> The machine:
>>>>  * Supermicro X9DRW 
>>>>  * Dual Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz
>>>>  * 128G RAM
>>>>  * Areca ARC-1882IX-16 (1G onboard cache)
>>>>
>>>> Nothing I have tried is effective:
>>>>  * Turning on BIOS above 4G decoding stops the Intel 10GBE ixgbe driver from functioning and doesn't fix the arcmsr
>>>>  * Unloading and reloading the arcmsr module from initramfs prompt
>>>>  * Downgrading the Areca 1882 bios to v1.52 as per http://faq.areca.com.tw/index.php?action=artikel&cat=7&id=902&artlang=en
>>>>  * Kernel parameters
>>>>  ** pci=nocrs 
>>>>  ** dom0_mem=8G 
>>>>  ** mem=3072M
>>>>  ** mem2048M cma=1024M
>>>>  ** cma=2048
>>>>  ** cma=3076 at 512M
>>>>  ** iommu=1 intel_iommu=1 
>>>>  ** arcmsr.host_can_queue=64 as per http://faq.areca.com.tw/index.php?action=artikel&cat=15&id=387&artlang=en
>>>>
>>>> I expected the arcmsr module to load and detect disks as it does with
>>>> the stock kernel.
>>>>
>>>> I can provide sysctl and dmesg output if it helps.
>>> Yes. The first thing needed is full startup logs, and for the Xen part
>>> preferably extra logging. In /etc/default/grub.d/xen.cfg in the
>>> GRUB_CMDLINE_XEN_DEFAULT setting, you can add loglvl=all, and then run
>>> update-grub and try to boot Xen+Linux again.
>>>
>>> Do you have a way to capture the logging during boot? Like, a working
>>> serial console or something similar?
>>>
>>> The output of dmesg when starting Linux without Xen is of course also
>>> interesting, so we can compare both scenarios.
>>>
>>> Hans
>> I tried using debian's paste https://paste.debian.net but it always
>> thought it was spam.
>>
>> dmesg output Xen Hypervisor 4.11 https://pastebin.com/3wUyYg0P
> This one shows a Linux kernel boot, not the Xen Hypervisor, which should
> go first (with all the (XEN) lines). By default the Xen output should
> show up on your (serial) console. If you do dmesg after starting Linux
> as dom0 after starting Xen, then you just get the Linux part of it.
>
> If it actually boots and it's usable to login and get a shell prompt
> etc, then you can immediately use xl dmesg to see the xen part, and if
> it doesn't, then you need to make sure you have some sort of serial
> console to capture the lines.
>
> To do a bug report upstream, we'll need that information.

Sorry, completely misunderstood.   Here is the output from the serial
terminal as Xen started.

https://paste.debian.net/1154644

Alex

>
>> dmesg output Debian Kernel 4.19.118-2+deb10u1 https://pastebin.com/GHzzW3vi
> K