[Xen-devel] Dealing with non-existent BDF devices in VT-d and in the hardware.

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

Hey,

I am one of those lucky folks who had purchased a motherboard that has bugs.

I figured I would post this email as way for a starting point
for some discussion on this - and perhaps have a similar as 'pci-phantom'
way of instructing the hypervisor what to do with them.

The problem I am seeing is that this device:

08:03.0 FireWire (IEEE 1394): Texas Instruments TSB43AB22A IEEE-1394a-2000 
Controller (PHY/Link) [iOHCI-Lynx]

Can't be passed in the guest. Or rather it can - but everytime
the guest (or domain0) tries to access I see:

(XEN) [VT-D]iommu.c:885: iommu_fault_status: Fault Overflow
(XEN) [VT-D]iommu.c:887: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]iommu.c:865: DMAR:[DMA Write] Request device [0000:08:00.0] fault 
addr 0, iommu reg = ffff82c3ffd53000
(XEN) DMAR:[fault reason 02h] Present bit in context entry is clear
(XEN) print_vtd_entries: iommu ffff83043dca99b0 dev 0000:08:00.0 gmfn 0
(XEN)     root_entry = ffff83043dc6b000
(XEN)     root_entry[8] = 3326b5001
(XEN)     context = ffff8303326b5000
(XEN)     context[0] = 0_0
(XEN)     ctxt_entry[0] not present


Of course the '08:00.0' device does not exist. It is rather this chipset:
07:00.0 PCI bridge: Tundra Semiconductor Corp. Device 8113 (rev 01)

that is buggy and using the wrong BDF when forwarding DMA requests from
devices underneath it (like this Firewire chip).

The hack I came up with was to create in the Xen code that deals with
PCI passthrough a copy of the bridge (so 07:00.0) but with a new
BDF: 08:00.0. And link it to the PCI device that I am passing to the
guest (so 08:03.0).

The end result is that when loading the driver (hack.c) one should
see:

(XEN) 0000:08:00.0 linked with 08:03.0
(XEN) [VT-D]iommu.c:1456: d0:PCI: map 0000:08:00.0
(XEN) [VT-D]iommu.c:1476: d0:PCI: map 0000:08:03.0
(XEN) PCI add link 0000:08:00.0

And when launching a guest with the BDF:
pci = ["08:03.0"]

the hypervisor will automatically also create an VT-d context for the
08:00.0 device.

To use this hack, apply the 
0001-xen-pci-Introduce-a-way-to-deal-with-buggy-hardware-.patch
to your hypervisor, compile and install.

And also compile the 'hack.c' module. There is an attached 'Makefile'
that will do it for you. Make sure you edit it to set the right BDF
entries in it.

Once done install your new hypervisor, and insmod ./hack.ko and try
passing in the device to your guest (or use it normally). The
'DMAR:[DMA Write]' error should go away.

This should be generic enough for most devices. It needn't be a bridge
that is spewing out these DMAR errors.

Attachment: 0001-xen-pci-Introduce-a-way-to-deal-with-buggy-hardware-.patch
Description: Text document

Attachment: hack.c
Description: Text document

Attachment: Makefile
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel