[Pkg-xen-devel] Bug#721345: xcp-xapi: xl pci-attach is called by xapi in a PCI passthrough attempt. An error is raised due to the use of xapi and the PCI device cannot be attached to the VM.

[Vangelis Tasoulas]

Package: xcp-xapi
Version: 1.3.2-15
Severity: normal

I am trying to pass an SRIOV virtual function in a virtual machine but the attempt fails with the following errors shown in the xcp-xapi.log

[20130829T19:29:17.909Z|debug|sriov1|314 UNIX /var/lib/xcp/xapi|VM.start R:9e8e10bd31bb|pciops] PCI devices from other-config:pci to attach: 0/0000:04:00.1
[20130829T19:29:18.012Z|debug|sriov1|314 UNIX /var/lib/xcp/xapi|VM.start R:9e8e10bd31bb|xenops] Doing FLR on pci device: 0000:04:00.1
[20130829T19:29:18.022Z|debug|sriov1|314 UNIX /var/lib/xcp/xapi|VM.start R:9e8e10bd31bb|xapi] Raised at forkhelpers.ml:181.30-76 -> pervasiveext.ml:22.2-9
[20130829T19:29:18.032Z|debug|sriov1|314 UNIX /var/lib/xcp/xapi|VM.start R:9e8e10bd31bb|xapi] Raised at forkhelpers.ml:181.30-76 -> pervasiveext.ml:22.2-9
[20130829T19:29:18.074Z|debug|sriov1|314 UNIX /var/lib/xcp/xapi|VM.start R:9e8e10bd31bb|xapi] Raised at forkhelpers.ml:181.30-76 -> pervasiveext.ml:22.2-9
[20130829T19:29:18.074Z|debug|sriov1|314 UNIX /var/lib/xcp/xapi|VM.start R:9e8e10bd31bb|xenops] xl pci-attach: Forkhelpers.Spawn_internal_error("ERROR:  A different toolstack (xapi) have been selected!\n", "", _)
[20130829T19:29:18.075Z|debug|sriov1|314 UNIX /var/lib/xcp/xapi|VM.start R:9e8e10bd31bb|backtrace] Raised at device.ml:1024.8-35 -> list.ml:69.12-15 -> helpers.ml:34.35-38
[20130829T19:29:18.075Z|debug|sriov1|314 UNIX /var/lib/xcp/xapi|VM.start R:9e8e10bd31bb|helpers] Ignoring exception: INTERNAL_ERROR: [ Device.PCI.Cannot_add(_, _) ] while attach_pcis

When trying for a second time I get the following extra line which shows that the first try has already bounded the virtual function to xen-pciback module:

[20130829T19:38:31.141Z|debug|sriov1|601 UNIX /var/lib/xcp/xapi|VM.start R:4a91dd3ec461|pciops] PCI devices from other-config:pci to attach: 0/0000:04:00.1
[20130829T19:38:31.141Z|debug|sriov1|601 UNIX /var/lib/xcp/xapi|VM.start R:4a91dd3ec461|xenops] pci: device 0000:04:00.1 already bounded to pciback
[20130829T19:38:31.141Z|debug|sriov1|601 UNIX /var/lib/xcp/xapi|VM.start R:4a91dd3ec461|xenops] Doing FLR on pci device: 0000:04:00.1
[20130829T19:38:31.156Z|debug|sriov1|601 UNIX /var/lib/xcp/xapi|VM.start R:4a91dd3ec461|xapi] Raised at forkhelpers.ml:181.30-76 -> pervasiveext.ml:22.2-9
[20130829T19:38:31.166Z|debug|sriov1|601 UNIX /var/lib/xcp/xapi|VM.start R:4a91dd3ec461|xapi] Raised at forkhelpers.ml:181.30-76 -> pervasiveext.ml:22.2-9
[20130829T19:38:31.190Z|debug|sriov1|601 UNIX /var/lib/xcp/xapi|VM.start R:4a91dd3ec461|xapi] Raised at forkhelpers.ml:181.30-76 -> pervasiveext.ml:22.2-9
[20130829T19:38:31.190Z|debug|sriov1|601 UNIX /var/lib/xcp/xapi|VM.start R:4a91dd3ec461|xenops] xl pci-attach: Forkhelpers.Spawn_internal_error("ERROR:  A different toolstack (xapi) have been selected!\n", "", _)
[20130829T19:38:31.190Z|debug|sriov1|601 UNIX /var/lib/xcp/xapi|VM.start R:4a91dd3ec461|backtrace] Raised at device.ml:1024.8-35 -> list.ml:69.12-15 -> helpers.ml:34.35-38
[20130829T19:38:31.190Z|debug|sriov1|601 UNIX /var/lib/xcp/xapi|VM.start R:4a91dd3ec461|helpers] Ignoring exception: INTERNAL_ERROR: [ Device.PCI.Cannot_add(_, _) ] while attach_pcis

What I did to make it work is to change the default toolstack from xapi to xl in the file /etc/default/xen, then I ran the "xl pci-attach <vm-name> 0000:04:00.1" manually and changed the toolstack back to xapi.
Eventually the pci device shows in the VM.


-- System Information:
Debian Release: 7.1
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.5.7-amd64 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages xcp-xapi depends on:
ii  hwdata                                         0.234-1
ii  libc6                                          2.13-38
ii  libpam0g                                       1.1.3-7.1
ii  libuuid1                                       2.20.1-5.3
ii  libvhd0                                        2.0.90-1
ii  libxen-4.1                                     4.1.4-3+deb7u1
ii  libxenstore3.0                                 4.1.4-3+deb7u1
ii  lsb-base                                       4.1+Debian8+deb7u1
ii  pciutils                                       1:3.1.9-6
ii  python                                         2.7.3-4
ii  python-xenapi                                  1.3.2-15
ii  stunnel4 [stunnel]                             3:4.53-1.1
ii  xcp-eliloader                                  0.1-4
ii  xcp-fe                                         0.5.2-3+b1
ii  xcp-networkd                                   1.3.2-15
ii  xcp-squeezed                                   1.3.2-15
ii  xcp-storage-managers                           0.1.1-3
ii  xcp-v6d                                        1.3.2-15
ii  xcp-xe                                         1.3.2-15
ii  xen-hypervisor-4.1-amd64 [xen-hypervisor-4.1]  4.1.4-3+deb7u1
ii  xen-utils-4.1                                  4.1.4-3+deb7u1
ii  zlib1g                                         1:1.2.7.dfsg-13

Versions of packages xcp-xapi recommends:
ii  cifs-utils           2:5.5-1
ii  xcp-guest-templates  0.1-4
ii  xcp-vncterm          0.1-2

xcp-xapi suggests no packages.

-- no debconf information