Re: [Xen-users] Does anybody in the Xen community have any experience with the xen-pciback driver and MSI-X??

[Matthew Hook <matthew.hook@xxxxxxxx>]

Greetings Andrew,

I have managed to get SR-IOV working to a degree under Xen but it required quite a late kernel build.  It seems to kinda work with kernel 3.11.2. I also get this error.

pciback 0000:06:00.1: seizing device

Oct  1 01:10:56 xen4 kernel: [   77.196621] pciback 0000:06:00.1: enabling device (0000 -> 0002)

Oct  1 01:10:56 xen4 kernel: [   77.196711] pciback 0000:06:00.1: MSI-X preparation failed (-38)

The other thing I found was that you need to use the latest OFED stack 3.5.

If you're using Ubuntu there are some PPA's for that.

I got these errors in the opensm log which I can't find any information about.

Oct 01 02:07:37 965995 [4590C700] 0x01 -> validate_requested_mgid: ERR 1B01: Wrong MGID Prefix 0x8000 must be 0xFF

Oct 01 02:07:37 966024 [4590C700] 0x01 -> mcmr_rcv_create_new_mgrp: ERR 1B22: Invalid requested MGID

Oct 01 02:07:37 966346 [4A115700] 0x01 -> validate_requested_mgid: ERR 1B01: Wrong MGID Prefix 0x8000 must be 0xFF

Oct 01 02:07:37 966394 [4A115700] 0x01 -> mcmr_rcv_create_new_mgrp: ERR 1B22: Invalid requested MGID

I wonder if this is related to msi-x and pciback.

Regards, Matt

> Greetings Xen user community,
>
> I am interested in using Mellanox ConnectX cards with SR-IOV capabilities to passthrough pci-e Virtual Functions (VFs) to Xen guests. The hope is to allow for the use of InfiniBand directly within virtual machines and thereby enable a plethora of high performance computing applications that already leverage InfiniBand interconnects. However, I have run into some issues using the xen-pciback driver and its initialization of MSI-X as required for VFs in Xen.  The hardware used is Mellanox Connect X3 MT27500 VPI pci-express cards set up in InfiniBand mode in HP blades with Intel Xeon E5-2670 CPUs and 42GB of memory.  SR-IOV is enabled in the system BIOS along with VT-X, and of course VT-d.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users