Problem with PCI-passthrough to PV guest

[Jürgen Groß <jgross@xxxxxxxx>]

SUSE QA is seeing a problem with PCI-passthrough of a SR-IOV to a PV guest
running a 6.4 based kernel, but I can reproduce the problem with upstream
kernel, too.

The guest is configured with "e820_host=1", but the PCI region on the host
isn't marked as "reserved" in the memory map, so it won't be reserved in
the either.

As the guest doesn't have the ACPI table available describing the PCI regions,
/proc/iomem in the guest won't include those regions as used, resulting in
the kernel's resource management to use those areas for potential memory:

 resource: avoiding allocation from e820 entry [mem 0x78edc000-0x79868fff]
 resource: avoiding allocation from e820 entry [mem 0x79d2a000-0x8fffffff]
 resource: remaining [mem 0x0000000090000000-0x00000000ffdfffff] available
 resource: avoiding allocation from e820 entry [mem 0xc7ffc000-0xc7ffcfff]
 resource: remaining [mem 0x0000000090000000-0x00000000c7ffbfff] available

dom0 /proc/iomem:
 ...
 80000000-8fffffff : PCI MMCONFIG 0000 [bus 00-ff]
 90000000-c7ffbfff : PCI Bus 0000:00    ← PCI MMIO window begins here
   90000000-900fffff : PCI Bus 0000:01  ← I350 VFs assigned in this range
   c6000000-c70fffff : PCI Bus 0000:04

dom0 e820 map:
 ...
 Xen: [mem 0x0000000079869000-0x0000000079d29fff] ACPI NVS
 Xen: [mem 0x0000000079d2a000-0x000000008fffffff] reserved
 Xen: [mem 0x00000000c7ffc000-0x00000000c7ffcfff] reserved
 Xen: [mem 0x00000000fbffc000-0x00000000fbffcfff] reserved
 ...

domU /proc/iomem:
 ...
 00100000-78f06fff : System RAM
   01000000-01ffffff : Kernel code
   ...
 90000000-97ffffff : System RAM
 fee00000-fee00fff : Local APIC

domU e820 map:
 ...
 Xen: [mem 0x0000000079869000-0x0000000079d29fff] ACPI NVS
 Xen: [mem 0x0000000079d2a000-0x000000008fffffff] reserved
 Xen: [mem 0x00000000c7ffc000-0x00000000c7ffcfff] reserved
 Xen: [mem 0x00000000fbffc000-0x00000000fbffcfff] reserved
 ...

The VF is showing up near 0x90000
 pci 0000:00:00.4: [8086:1520] type 00 class 0x020000
 pci 0000:00:00.4: reg 0x10: [mem 0x90004000-0x90007fff 64bit pref]
 pci 0000:00:00.4: reg 0x1c: [mem 0x90024000-0x90027fff 64bit pref]
 pcifront pci-0: New device on 0000:00:00.4 found.
 pcifront pci-0: claiming resource 0000:00:00.4/0
 pci 0000:00:00.4: can't claim BAR 0 [mem 0x90004000-0x90007fff 64bit pref]: 
address conflict with System RAM [mem 0x90000000-0x97ffffff]
 pcifront pci-0: Could not claim resource 0000:00:00.4/0! Device offline. Try 
using e820_host=1 in the guest config.
 pcifront pci-0: claiming resource 0000:00:00.4/3
 pci 0000:00:00.4: can't claim BAR 3 [mem 0x90024000-0x90027fff 64bit pref]: 
address conflict with System RAM [mem 0x90000000-0x97ffffff]
 pcifront pci-0: Could not claim resource 0000:00:00.4/3! Device offline. Try 
using e820_host=1 in the guest config.
My first idea for solving this was to add the PCI regions from dom0's
/proc/iomem to the e820 map of the guest, but this is more a hack than a sane
solution.

Thoughts?


Juergen

Attachment: OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature.asc
Description: OpenPGP digital signature