[Pkg-xen-devel] Bug#693154: xen-hypervisor-4.0-amd64: Xen "map irq failed" with Intel igb driver and 82576 quad port nic

Matt Baker matt.baker at bristol.ac.uk
Tue Nov 13 17:43:48 UTC 2012 

Package: xen-hypervisor-4.0-amd64
Version: 4.0.1-5.4
Severity: important


When using the the intel igb driver from the 3.2.0-0.bpo.4-amd64 kernel and debian squeeze hypervisor with a Intel 82576 quad port nic the first nic fails to get an IRQ mapping:

relevant lines from dmesg

----8<----
[   24.264857] Intel(R) Gigabit Ethernet Network Driver - version 3.2.10-k
[   24.264929] Copyright (c) 2007-2011 Intel Corporation.
[   24.265068] xen: registering gsi 55 triggering 0 polarity 1
[   24.265087] xen map irq failed -22
[   24.265158] Failed to setup GSI :55, err_code:-22
[   24.265250] igb 0000:06:00.0: PCI INT A: failed to register GSI
[   24.265339] igb: probe of 0000:06:00.0 failed with error -1
[   24.265436] xen: registering gsi 49 triggering 0 polarity 1
[   24.265466] xen: --> pirq=49 -> irq=49 (gsi=49)
[   24.265476] igb 0000:06:00.1: PCI INT B -> GSI 49 (level, low) -> IRQ 49
[   24.265607] igb 0000:06:00.1: setting latency timer to 64
----8<----
[   24.460711] igb 0000:06:00.1: Intel(R) Gigabit Ethernet Network Connection
[   24.460787] igb 0000:06:00.1: eth2: (PCIe:2.5Gb/s:Width x4) 00:1b:21:d5:d7:c1
[   24.461178] igb 0000:06:00.1: eth2: PBA No: G18771-002
[   24.461245] igb 0000:06:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
[   24.461354] xen: registering gsi 40 triggering 0 polarity 1
[   24.461377] xen: --> pirq=40 -> irq=40 (gsi=40)
[   24.461385] igb 0000:07:00.0: PCI INT A -> GSI 40 (level, low) -> IRQ 40
[   24.461476] igb 0000:07:00.0: setting latency timer to 64
[   24.607690] usb 1-3: new high-speed USB device number 2 using ehci_hcd
[   24.624737] scsi 0:0:32:0: Enclosure         DP       BACKPLANE        1.10 PQ: 0 ANSI: 5
[   24.643887] scsi 0:2:0:0: Direct-Access     DELL     PERC H700        2.10 PQ: 0 ANSI: 5
[   24.652706] igb 0000:07:00.0: Intel(R) Gigabit Ethernet Network Connection
[   24.652781] igb 0000:07:00.0: eth3: (PCIe:2.5Gb/s:Width x4) 00:1b:21:d5:d7:c4
[   24.653171] igb 0000:07:00.0: eth3: PBA No: G18771-002
[   24.653238] igb 0000:07:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
[   24.653343] xen: registering gsi 50 triggering 0 polarity 1
[   24.653361] xen: --> pirq=50 -> irq=50 (gsi=50)
[   24.653368] igb 0000:07:00.1: PCI INT B -> GSI 50 (level, low) -> IRQ 50
[   24.653455] igb 0000:07:00.1: setting latency timer to 64
[   24.666787] sd 0:2:0:0: [sda] 3511681024 512-byte logical blocks: (1.79 TB/1.63 TiB)
[   24.666955] sd 0:2:0:0: [sda] Write Protect is off
[   24.667022] sd 0:2:0:0: [sda] Mode Sense: 1f 00 00 08
[   24.667076] sd 0:2:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   24.667769]  sda: sda1 sda2 sda3
[   24.668375] sd 0:2:0:0: [sda] Attached SCSI disk
[   24.740067] usb 1-3: New USB device found, idVendor=0424, idProduct=2514
[   24.740141] usb 1-3: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[   24.740517] hub 1-3:1.0: USB hub found
[   24.740693] hub 1-3:1.0: 4 ports detected
[   24.844761] igb 0000:07:00.1: Intel(R) Gigabit Ethernet Network Connection
[   24.844836] igb 0000:07:00.1: eth4: (PCIe:2.5Gb/s:Width x4) 00:1b:21:d5:d7:c5
[   24.845228] igb 0000:07:00.1: eth4: PBA No: G18771-002
[   24.845294] igb 0000:07:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
----8<----

sysfs shows the interface in the devices/pci but no symlink to driver or a "net" directory:

root at boomer ~
$ ls -l /sys/devices/pci0000:00/0000:00:09.0/0000:04:00.0/0000:05:02.0/0000:06:00.*/
/sys/devices/pci0000:00/0000:00:09.0/0000:04:00.0/0000:05:02.0/0000:06:00.0/:
total 0
-rw-r--r-- 1 root root    4096 Nov 13 17:37 broken_parity_status
-r--r--r-- 1 root root    4096 Oct 16 15:25 class
-rw-r--r-- 1 root root    4096 Oct 16 15:25 config
-r--r--r-- 1 root root    4096 Nov 13 17:37 consistent_dma_mask_bits
-r--r--r-- 1 root root    4096 Oct 16 15:25 device
-r--r--r-- 1 root root    4096 Nov 13 17:37 dma_mask_bits
-rw------- 1 root root    4096 Nov 13 17:37 enable
-r--r--r-- 1 root root    4096 Oct 16 15:25 irq
-r--r--r-- 1 root root    4096 Nov 13 17:37 local_cpulist
-r--r--r-- 1 root root    4096 Nov 13 17:37 local_cpus
-r--r--r-- 1 root root    4096 Nov 13 17:37 modalias
-rw-r--r-- 1 root root    4096 Nov 13 17:37 msi_bus
-r--r--r-- 1 root root    4096 Nov 13 17:37 numa_node
drwxr-xr-x 2 root root       0 Nov 13 17:37 power
--w--w---- 1 root root    4096 Nov 13 17:37 remove
--w--w---- 1 root root    4096 Nov 13 17:37 rescan
--w------- 1 root root    4096 Nov 13 17:37 reset
-r--r--r-- 1 root root    4096 Oct 16 15:25 resource
-rw------- 1 root root  131072 Nov 13 17:37 resource0
-rw------- 1 root root 4194304 Nov 13 17:37 resource1
-rw------- 1 root root      32 Nov 13 17:37 resource2
-rw------- 1 root root   16384 Nov 13 17:37 resource3
-rw------- 1 root root 4194304 Nov 13 17:37 rom
lrwxrwxrwx 1 root root       0 Nov 13 17:37 subsystem -> ../../../../../../bus/pci
-r--r--r-- 1 root root    4096 Oct 16 15:25 subsystem_device
-r--r--r-- 1 root root    4096 Oct 16 15:25 subsystem_vendor
-rw-r--r-- 1 root root    4096 Nov 13 17:37 uevent
-r--r--r-- 1 root root    4096 Oct 16 15:25 vendor

/sys/devices/pci0000:00/0000:00:09.0/0000:04:00.0/0000:05:02.0/0000:06:00.1/:
total 0
-rw-r--r-- 1 root root    4096 Nov 13 17:37 broken_parity_status
-r--r--r-- 1 root root    4096 Oct 16 15:25 class
-rw-r--r-- 1 root root    4096 Oct 16 15:25 config
-r--r--r-- 1 root root    4096 Nov 13 17:37 consistent_dma_mask_bits
-r--r--r-- 1 root root    4096 Oct 16 15:13 device
-r--r--r-- 1 root root    4096 Nov 13 17:37 dma_mask_bits
lrwxrwxrwx 1 root root       0 Nov 13 14:29 driver -> ../../../../../../bus/pci/drivers/igb
-rw------- 1 root root    4096 Nov 13 17:37 enable
-r--r--r-- 1 root root    4096 Oct 16 15:25 irq
-r--r--r-- 1 root root    4096 Nov 13 17:37 local_cpulist
-r--r--r-- 1 root root    4096 Nov 13 17:37 local_cpus
-r--r--r-- 1 root root    4096 Nov 13 17:37 modalias
-rw-r--r-- 1 root root    4096 Nov 13 17:37 msi_bus
drwxr-xr-x 3 root root       0 Nov 13 14:29 net
-r--r--r-- 1 root root    4096 Nov 13 17:37 numa_node
drwxr-xr-x 2 root root       0 Nov 13 17:37 power
--w--w---- 1 root root    4096 Nov 13 17:37 remove
--w--w---- 1 root root    4096 Nov 13 17:37 rescan
--w------- 1 root root    4096 Nov 13 17:37 reset
-r--r--r-- 1 root root    4096 Oct 16 15:25 resource
-rw------- 1 root root  131072 Nov 13 17:37 resource0
-rw------- 1 root root 4194304 Nov 13 17:37 resource1
-rw------- 1 root root      32 Nov 13 17:37 resource2
-rw------- 1 root root   16384 Nov 13 17:37 resource3
-rw------- 1 root root 4194304 Nov 13 17:37 rom
lrwxrwxrwx 1 root root       0 Nov 13 14:29 subsystem -> ../../../../../../bus/pci
-r--r--r-- 1 root root    4096 Oct 16 15:25 subsystem_device
-r--r--r-- 1 root root    4096 Oct 16 15:25 subsystem_vendor
-rw-r--r-- 1 root root    4096 Nov 13 14:29 uevent
-r--r--r-- 1 root root    4096 Oct 16 15:13 vendor

When booted into 3.2.0-0.bpo.4-amd64 without Xen all 4 nics are detected and assigned GSI IRQ mappings using MSI-X.

I've tried using pci=nomsi and legacy interrupts are assigned but I still get the mapping error for the first nic.

Is this a Xen problem?

The server is a Dell R510.

-- System Information:
Debian Release: 6.0.6
  APT prefers stable-updates
  APT policy: (800, 'stable-updates'), (800, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.2.0-0.bpo.4-amd64 (SMP w/24 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

xen-hypervisor-4.0-amd64 depends on no packages.

Versions of packages xen-hypervisor-4.0-amd64 recommends:
ii  xen-utils-4.0                 4.0.1-5.4  XEN administrative tools

Versions of packages xen-hypervisor-4.0-amd64 suggests:
pn  xen-docs-4.0                  <none>     (no description available)

-- no debconf information

More information about the Pkg-xen-devel mailing list