[Xen-devel] Re: [pvops xen/next ][iommu] attenpt to passthrough PCI-e usb controllor to PV domU: (XEN) traps.c:2309:d1 Domain attempted WRMSR 000000000000008b from 00000a07:00000000 to 00000000:00000000.

[Sander Eikelenboom <linux@xxxxxxxxxxxxxx>]

Hello Weidong/Konrad,

1) With iommu=0, the DMAR fault is gone (of course), but
        > (XEN) traps.c:2309:d1 Domain attempted WRMSR 000000000000008b from 
00000a07:00000000 to 00000000:00000000.
        Stays in xm-dmesg, the pv guests is booted, and lspci shows the pci 
device. But the device doesn't function properly.


2) With iommu enabled and iommu_inclusive_mapping=1 and HVM:
        I can't start the HVM, it complains:
        Error: pci: to avoid potential security issue, 0000:03:00.0 is not 
allowed to be assigned to guest since it is behind PCIe switch that does not 
support or enable ACS.

        This USB controller does have multiple pci-e bridges of its own, as can 
be seen in the pci tree
        -[0000:00]-+-00.0
           +-01.0-[0000:01-04]----00.0-[0000:02-04]--+-01.0-[0000:03]----00.0   
 <-- The usb controller i try to passthrough
           |                                         \-02.0-[0000:04]----00.0
           +-02.0
           +-03.0
           +-19.0
           +-1a.0
           +-1a.1
           +-1a.2
           +-1a.7
           +-1b.0
           +-1c.0-[0000:06]--
           +-1c.4-[0000:05]----00.0
           +-1d.0
           +-1d.1
           +-1d.2
           +-1d.7
           +-1e.0-[0000:07]--+-00.0
           |                 \-03.0
           +-1f.0
           +-1f.2
           +-1f.3
           \-1f.5


         When i use pciback to hide those pci-e bridges, and try to pass them 
through as well,
         although lspci -k and dmesg show they are actually owned by pciback, 
starting the HVM fails because the pci-e bridges are supposedly not owned by 
pciback or stub.


Attached the xm-dmesg of boot without iommu

--
Sander






Monday, March 22, 2010, 9:04:09 AM, you wrote:

> The faults were caused by that the DMA address was not mapped in VT-d page 
> table.

> Could you have following two tries:
>         1) assign it to pv domU without VT-d
>         2) assign it to a hvm guest

> Regards,
> Weidong

> -----Original Message-----
> From: Sander Eikelenboom [mailto:linux@xxxxxxxxxxxxxx] 
> Sent: Monday, March 22, 2010 5:20 AM
> To: Konrad Rzeszutek Wilk
> Cc: Han, Weidong; xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: [pvops xen/next ][iommu] attenpt to passthrough PCI-e usb controllor 
> to PV domU: (XEN) traps.c:2309:d1 Domain attempted WRMSR 000000000000008b 
> from 00000a07:00000000 to 00000000:00000000.

> Hi Han/Konrad,

> In my setup i'm trying to passthrough an USB 3.0 pci-e controller  to a PV 
> domU.
> - xen: 4.0.0-rc6
> - dom0: kernel xen/next
> - domU: kernel 2.6.33 from 
> git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
>         ( to have pci-front together with most recent usb3.0 xhci drivers.


> - USB 3.0 xhci drivers work fine on the baremetal with the 2.6.33 kernel.

> This is on a intel Q45 chipset with IOMMU.

> This is my boot config:
> title           xen-4.0.0-rc6.gz / Debian GNU/Linux,  kernel 2.6.32
> root            (hd0,0)
> kernel          /boot/xen-4.0.0-rc6.gz dom0_mem=768M loglvl=all 
> loglvl_guest=all  iommu=pv iommu_inclusive_mapping=1
> module          /boot/vmlinuz-2.6.32 root=/dev/sda1 ro earlyprintk=xen 
> max_loop=255 xen-pciback.hide=(03:00.0)
> module          /boot/initrd.img-2.6.32

> When booting the domU xm dmesg gets filled with the following when the usb 
> controller tries to initialize/:

> (XEN) traps.c:2309:d1 Domain attempted WRMSR 000000000000008b from 
> 00000a07:00000000 to 00000000:00000000.
> (XEN) [VT-D]iommu.c:821: iommu_fault_status: Primary Pending Fault
> (XEN) [VT-D]iommu.c:796: DMAR:[DMA Read] Request device [03:00.0] fault addr 
> 1ff94000, iommu reg = ffff82c3fff54000
> (XEN) DMAR:[fault reason 06h] PTE Read access is not set
> (XEN) print_vtd_entries: iommu = ffff83007c866970 bdf = 3:0.0 gmfn = 1ff94
> (XEN)     root_entry = ffff83007c872000
> (XEN)     root_entry[3] = 78f56001
> (XEN)     context = ffff830078f56000
> (XEN)     context[0] = 101_2f0e1001
> (XEN)     l3 = ffff83002f0e1000
> (XEN)     l3_index = 0
> (XEN)     l3[0] = 2f0e0003
> (XEN)     l2 = ffff83002f0e0000
> (XEN)     l2_index = ff
> (XEN)     l2[ff] = 0
> (XEN)     l2[ff] not present



> Anyone any tips on what i could try ?, is this something caused by xen, or 
> something by the usb driver not adhering to kernel DMA-api ?

> Attached:

> - xm-info.txt
> - xm-dmesg.txt
> - xend.log

> - dom0-dmesg.txt
> - dom0-lspci-tree.txt
> - dom0-lspci.txt

> - domU-lspci.txt
> - domU-dmesg.txt







-- 
Best regards,
 Sander                            mailto:linux@xxxxxxxxxxxxxx

Attachment: xm-dmesg-no-iommu.txt
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel