[Xen-devel] Re: MSI 890FXA-GD70: AMD IOMMU, VmError: Failed to assign device to IOMMU, (XEN) AMD_IOV: IO_PAGE_FALT: domain:10, device id:0x900, fault address:0x0

[Wei Wang2 <wei.wang2@xxxxxxx>]

Hi,
AFAIK, Xen does not allow devices which share a same device id to be assigned 
into different domains. According to the IVRS table, device 03:xx.x uses the 
device id of 00:14.4, which means, both 03:xx.x and 00:14.4 should be hidden 
from dom0 and them can only be assign to a same domain.

Device  00:14.2 might be an different story. Probably, current xen had added 
some checks to prevent devices on the SAME device number from been spread 
among different domains. I suppose you did not hide all functions on bus 0 
device 14?

Thanks,
Wei

On Wednesday 14 July 2010 01:13:40 Huang2, Wei wrote:
> [cc Wei Wang here]
>
> In the function reassign_device() in pci_amd_iommu.c, it seems
> pci_get_pdev_by_domain() fail to return a correct pci_dev. Here is the
> decode of IVRS table. It looks like 03:xx.x is behind a bridge 00:14.4 and
> uses this value as an alias ID. That might be a cause. From IVRS, I don't
> see anything wrong with 00:14.2 though. Maybe you can add more debug code
> to reassign_device().
>
> -Wei
>
> 02 10 00 00                         DEV=00:2.0
> 03 00 0f 00 04 01 0f 00             DEV=[0f:1.0, 0f:1.0]
> 02 18 00 00                         DEV=00:2.0
> 02 00 0e 00                         DEV=0e:0.0
> 02 20 00 00                         DEV=00:4.0
> 02 00 0d 00                         DEV=0d:0.0
> 02 28 00 00                         DEV=00:5.0
> 02 00 0c 00                         DEV=0c:0.0
> 02 30 00 00                         DEV=00:6.0
> 02 00 0b 00                         DEV=0b:0.0
> 02 48 00 00                         DEV=00:9.0
> 02 00 0a 00                         DEV=0a:0.0
> 02 50 00 00                         DEV=00:a.0
> 02 00 08 00 00 00 00 00             DEV=08:0.0
> 43 08 09 00 00 00 09 00 04 ff 09 00 DEV=[09:1.0, 09:1f.7]
> 02 58 00 00                         DEV=00:b.0
> 02 00 04 00                         DEV=04:0.0
> 02 08 05 00                         DEV=05:1.0
> 02 00 07 00                         DEV=07:0.0
> 02 10 05 00                         DEV=05:2.0
> 03 00 06 00 04 00 06 00             DEV=06:0.0
> 02 88 00 00                         DEV=00:11.0
> 03 90 00 00 04 92 00 00             DEV=[00:12.0, 00:12.2]
> 03 98 00 00 04 9a 00 00             DEV=[00:13.0, 00:13.2]
> 02 a0 00 d7                         DEV=00:14.0
> 02 a1 00 00                         DEV=00:14.1
> 02 a2 00 00                         DEV=00:14.2
> 02 a3 00 00                         DEV=00:14.3
> 02 a4 00 00                         DEV=00:14.4
> 43 00 03 00 00 a4 00 00 04 ff 03 00 DEV=[03:0.0, 03:1f.7]
> 02 a5 00 00                         DEV=00:14.5
> 02 a8 00 00                         DEV=00:16.0
> 02 a9 00 00                         DEV=00:16.1
> 02 00 01 00                         DEV=01:0.0
> 03 b0 00 00 04 b2 00 00 00 00 00 00 DEV=[00:16.0, 00:16.2]
> 48 00 00 d7 00 a0 00 02
> 48 00 00 00 07 01 00 01
>
>
>
> -----Original Message-----
> From: Sander Eikelenboom [mailto:linux@xxxxxxxxxxxxxx]
> Sent: Tuesday, July 13, 2010 5:07 PM
> To: xen-devel@xxxxxxxxxxxxxxxxxxx
> Cc: Przywara, Andre; Joerg Roedel; Huang2, Wei; Konrad Rzeszutek Wilk;
> Jeremy Fitzhardinge Subject: MSI 890FXA-GD70: AMD IOMMU, VmError: Failed to
> assign device to IOMMU, (XEN) AMD_IOV: IO_PAGE_FALT: domain:10, device
> id:0x900, fault address:0x0
>
> Hi All,
>
> I have bought a MSI 890FXA-GD70, and have been able to enable the IOMMU
> with a beta-bios from MSI. I succeed in passing through some pci-e devices
> to domU's (PV), but some other devices fail with both PV and HVM domains.
>
> The message is allways the same "Failed to assign device to IOMMU", with
> some added printk's the return code from "iommu.c/assign_device" seems to
> be ffffffed instead of the desired 0.
>
> Devices 00:14.2 and 03:06.* cause the domain(PV and HVM) to fail on
> creation with: Failed to assign device to IOMMU (BDF) Devices 09:01.* and
> 07:00.0 passthrough fine ( visible and working in pv domU).
>
> Since it's a beta bios, could it be the IVRS table supplied by the bios is
> not complete and therefore the iommu functions only for some devices ? With
> amd_iommu_debug it sometimes also prints an "IO_PAGE_FALT":
>          (XEN) [VT-D]iommu.c:144: SEIK: assign_device: domain 14 (0:14.2)!
>          (XEN) [VT-D]iommu.c:162: SEIK: assign_device rc: ffffffed !
>          (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device
> (0:14.2) failed (XEN) AMD_IOV: IO_PAGE_FALT: domain:10, device id:0x900,
> fault address:0x0
>
>
> --
> Sander
>
>
>
>
>
>
>
> Xen version: xen-4.0-testing-pre4 (changeset:   21263:57859775f88f)
> Dom0 kernel: 2.6.32.16 pvops (commit
> 006bfe8f594370f36f09d8e8ab13692d34e88533)
>
> Attached:
> - xm info
> - xm dmesg
> - dmesg
> - lspci -vvvknn
> - Dump of ACPI IVRS table
> - Dump of ACPI DSDT table
>
>
> serveerstertje:~/iommu-report# xm pci-list-assignable-devices 0000:0e:00.0
> 0000:09:01.0 0000:09:01.1
> 0000:09:01.2
> 0000:03:06.0
> 0000:03:06.1
> 0000:03:06.2
> 0000:00:14.2
>
>
> grub menu.lst:
> title           xen-4.0.1-rc4-pre.gz / Debian GNU/Linux,  kernel
> 2.6.32.16+xen-2.6.32.x-20100713-a root            (hd0,0)
> kernel          /xen-4.0.1-rc4-pre.gz dom0_mem=768M loglvl=all
> loglvl_guest=all com1=115200,8n1 console=com1,vga
> iommu=1,pv,verbose,amd_iommu_debug module         
> /vmlinuz-2.6.32.16+xen-2.6.32.x-20100713-a
> root=/dev/mapper/serveerstertje-root ro earlyprintk=xen max_loop=255
> loop_max_part=63
> xen-pciback.hide=(00:14.2)(03:06.0)(03:06.1)(03:06.2)(07:00.0)(09:01.0)(09:
>01.1)(09:01.2)(0e:00.0)
> pci=resource_alignment=00:14.2;03:06.0;03:06.1;03:06.2;09:01.0;09:01.1;09:0
>1.2 module          /initrd.img-2.6.32.16+xen-2.6.32.x-20100713-a
>
>
>
>
> lspci:
>
> 00:00.0 Host bridge: ATI Technologies Inc RD890 Northbridge only single
> slot PCI-e GFX Hydra part (rev 02) 00:00.2 Generic system peripheral
> [0806]: ATI Technologies Inc Device 5a23 00:02.0 PCI bridge: ATI
> Technologies Inc RD890 PCI to PCI bridge (PCI express gpp port B) 00:03.0
> PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge (PCI express gpp
> port C) 00:04.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge
> (PCI express gpp port D) 00:05.0 PCI bridge: ATI Technologies Inc RD890 PCI
> to PCI bridge (PCI express gpp port E) 00:06.0 PCI bridge: ATI Technologies
> Inc RD890 PCI to PCI bridge (PCI express gpp port F) 00:09.0 PCI bridge:
> ATI Technologies Inc RD890 PCI to PCI bridge (PCI express gpp port H)
> 00:0a.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge (external
> gfx1 port A) 00:0b.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI
> bridge (NB-SB link) 00:11.0 SATA controller: ATI Technologies Inc
> SB700/SB800 SATA Controller [IDE mode] (rev 40) 00:12.0 USB Controller: ATI
> Technologies Inc SB700/SB800 USB OHCI0 Controller 00:12.2 USB Controller:
> ATI Technologies Inc SB700/SB800 USB EHCI Controller 00:13.0 USB
> Controller: ATI Technologies Inc SB700/SB800 USB OHCI0 Controller 00:13.2
> USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller
> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 41) 00:14.1
> IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller (rev 40)
> 00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA) (rev
> 40) 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host
> controller (rev 40) 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to
> PCI Bridge (rev 40) 00:14.5 USB Controller: ATI Technologies Inc
> SB700/SB800 USB OHCI2 Controller 00:15.0 PCI bridge: ATI Technologies Inc
> Device 43a0 00:16.0 USB Controller: ATI Technologies Inc SB700/SB800 USB
> OHCI0 Controller 00:16.2 USB Controller: ATI Technologies Inc SB700/SB800
> USB EHCI Controller 00:18.0 Host bridge: Advanced Micro Devices [AMD]
> Family 10h [Opteron, Athlon64, Sempron] HyperTransport Configuration
> 00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 10h [Opteron,
> Athlon64, Sempron] Address Map 00:18.2 Host bridge: Advanced Micro Devices
> [AMD] Family 10h [Opteron, Athlon64, Sempron] DRAM Controller 00:18.3 Host
> bridge: Advanced Micro Devices [AMD] Family 10h [Opteron, Athlon64,
> Sempron] Miscellaneous Control 00:18.4 Host bridge: Advanced Micro Devices
> [AMD] Family 10h [Opteron, Athlon64, Sempron] Link Control 03:06.0 USB
> Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev
> 62) 03:06.1 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1
> Controller (rev 62) 03:06.2 USB Controller: VIA Technologies, Inc. USB 2.0
> (rev 65) 04:00.0 PCI bridge: PLX Technology, Inc. Device 8613 (rev ba)
> 05:01.0 PCI bridge: PLX Technology, Inc. Device 8613 (rev ba) 05:02.0 PCI
> bridge: PLX Technology, Inc. Device 8613 (rev ba) 06:00.0 SATA controller:
> Device 1b4b:9123 (rev 10) 07:00.0 USB Controller: NEC Corporation Device
> 0194 (rev 03) 08:00.0 PCI bridge: Texas Instruments XIO2000(A)/XIO2200(A)
> PCI Express-to-PCI Bridge (rev 03) 09:01.0 USB Controller: NEC Corporation
> USB (rev 43) 09:01.1 USB Controller: NEC Corporation USB (rev 43)
> 09:01.2 USB Controller: NEC Corporation USB 2.0 (rev 04) 0a:00.0 USB
> Controller: NEC Corporation Device 0194 (rev 03) 0b:00.0 Ethernet
> controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express
> Gigabit Ethernet controller (rev 03) 0c:00.0 Ethernet controller: Realtek
> Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet
> controller (rev 03) 0d:00.0 FireWire (IEEE 1394): VIA Technologies, Inc.
> Device 3403 0e:00.0 VGA compatible controller: nVidia Corporation GeForce
> 8400 GS (rev a1) 0f:00.0 VGA compatible controller: ATI Technologies Inc
> Mobility Radeon HD 3450 0f:00.1 Audio device: ATI Technologies Inc RV620
> Audio device [Radeon HD 34xx Series]




_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel