pci passthrough and msi-x

[Henrik Riomar <rio@xxxxxxxxx>]

Hi,

What options are needed to get MSI-X interrupts working with pci passthrough?

Running Linux 5.10 in dom0 and in the guest with XEN 4.14.1.

Passing trough two Intel X553 NICs to the guest with this in xl.conf:
pci         = [ '08:00.0','08:00.1' ]

But what else is needed for MSI-X?

Also what about the swiotlb warning? whats needed to suppress fix that?

Thanks,
 Henrik

dmesg in domU:
[    1.509720] Mounting root: ok.
[    2.614502] cryptd: max_cpu_qlen set to 1000
[    2.619136] SSE version of gcm_enc/dec engaged.
[    2.632522] EDAC pnd2: Failed to register device with error -19.
[    2.667770] input: PC Speaker as /devices/platform/pcspkr/input/input0
[    2.702441] pcifront pci-0: Installing PCI frontend
[    2.703239] xen:swiotlb_xen: Warning: only able to allocate 4 MB for 
software IO TLB
[    2.767077] software IO TLB: mapped [mem 
0x0000000005800000-0x0000000005c00000] (4MB)
[    2.767384] pcifront pci-0: Creating PCI Frontend Bus 0000:00
[    2.767453] pcifront pci-0: PCI host bridge to bus 0000:00
[    2.767457] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    2.767459] pci_bus 0000:00: root bus resource [mem 0x00000000-0x7fffffffff]
[    2.767461] pci_bus 0000:00: root bus resource [bus 00-ff]
[    2.767869] pci 0000:00:00.0: [8086:15e4] type 00 class 0x020000
[    2.768544] pci 0000:00:00.0: reg 0x10: [mem 0xdd600000-0xdd7fffff 64bit 
pref]
[    2.769524] pci 0000:00:00.0: reg 0x20: [mem 0xdd804000-0xdd807fff 64bit 
pref]
[    2.769857] pci 0000:00:00.0: reg 0x30: [mem 0xdf180000-0xdf1fffff pref]
[    2.771631] pci 0000:00:00.0: [Firmware Bug]: reg 0x184: invalid BAR (can't 
size)
[    2.772607] pci 0000:00:00.0: [Firmware Bug]: reg 0x190: invalid BAR (can't 
size)
[    2.780194] pci 0000:00:00.1: [8086:15e4] type 00 class 0x020000
[    2.780845] pci 0000:00:00.1: reg 0x10: [mem 0xdd400000-0xdd5fffff 64bit 
pref]
[    2.781816] pci 0000:00:00.1: reg 0x20: [mem 0xdd800000-0xdd803fff 64bit 
pref]
[    2.782139] pci 0000:00:00.1: reg 0x30: [mem 0xdf100000-0xdf17ffff pref]
[    2.783949] pci 0000:00:00.1: [Firmware Bug]: reg 0x184: invalid BAR (can't 
size)
[    2.784917] pci 0000:00:00.1: [Firmware Bug]: reg 0x190: invalid BAR (can't 
size)
[    2.795870] pcifront pci-0: claiming resource 0000:00:00.0/0
[    2.795875] pcifront pci-0: claiming resource 0000:00:00.0/4
[    2.795876] pcifront pci-0: claiming resource 0000:00:00.0/6
[    2.795878] pcifront pci-0: claiming resource 0000:00:00.1/0
[    2.795879] pcifront pci-0: claiming resource 0000:00:00.1/4
[    2.795883] pcifront pci-0: claiming resource 0000:00:00.1/6
[    2.799838] dca service started, version 1.12.1
[    2.841452] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver
[    2.841455] ixgbe: Copyright (c) 1999-2016 Intel Corporation.
[    2.853935] ixgbe 0000:00:00.0: Xen PCI mapped GSI17 to IRQ19
[    3.097931] EDAC pnd2: Failed to register device with error -19.
[    3.193603] Interrupt reservation exceeds available resources
[    3.193683] ixgbe 0000:00:00.0: Failed to allocate MSI-X interrupts. Err: -28
[    3.193686] ixgbe 0000:00:00.0: Disabling SR-IOV support
[    3.193688] ixgbe 0000:00:00.0: Disabling RSS support
[    3.193901] Interrupt reservation exceeds available resources
[    3.193912] ixgbe 0000:00:00.0: Failed to allocate MSI interrupt, falling 
back to legacy. Error: -28
[    3.193918] ixgbe 0000:00:00.0: Multiqueue Disabled: Rx Queue count = 1, Tx 
Queue count = 1 XDP Queue count = 0
[    3.321556] ixgbe 0000:00:00.0: MAC: 6, PHY: 27, PBA No: 030000-000
[    3.321560] ixgbe 0000:00:00.0: 3c:ec:ef:03:0b:d6
[    3.369729] ixgbe 0000:00:00.0: Intel(R) 10 Gigabit Network Connection
[    3.373883] ixgbe 0000:00:00.1: Xen PCI mapped GSI18 to IRQ20
[    3.903506] Interrupt reservation exceeds available resources
[    3.903566] ixgbe 0000:00:00.1: Failed to allocate MSI-X interrupts. Err: -28
[    3.903569] ixgbe 0000:00:00.1: Disabling SR-IOV support
[    3.903570] ixgbe 0000:00:00.1: Disabling RSS support
[    3.903800] Interrupt reservation exceeds available resources
[    3.903811] ixgbe 0000:00:00.1: Failed to allocate MSI interrupt, falling 
back to legacy. Error: -28
[    3.903825] ixgbe 0000:00:00.1: Multiqueue Disabled: Rx Queue count = 1, Tx 
Queue count = 1 XDP Queue count = 0
[    4.036582] ixgbe 0000:00:00.1: MAC: 6, PHY: 27, PBA No: 030000-000
[    4.036587] ixgbe 0000:00:00.1: 3c:ec:ef:03:0b:d7
[    4.080368] ixgbe 0000:00:00.1: Intel(R) 10 Gigabit Network Connection