Re: [Xen-users] vga passthrogh on mainboards with broken ivrs tables

[Michaael Mair-Keimberger <m.mairkeimberger@xxxxxxxxx>]

Hi Eric,

I still couldn't get it to work. However, since the ivrs_ioapic kernel
parameter didn't do anything (i didn't even got any log messages about
it) i've dug further into this parameter.
Sadly i found out it's not included in xen-4.3.1 and since gentoo
doesn't provide git ebuilds for xen i'll gonna wait until next xen
will be released. xen-4.4 should be release on 21.01.14 - Thats not that
far away and i guess i can wait for it :)

If you want to passthrough your graphic card i suggest you to install a older
version of xen. Personally i have my own xen-4.2.1 ebuild which don't
include XSA-36. That one works perfectly well.
If irvs_ioapic won't help with next xen I'll go back too...

Regards,

Michael

On Sun, Dec 22, 2013 at 06:46:29PM -0700, Eric Houby wrote:
> Michael,
> 
> I have a GA-990FXA-UD3 H/W ver 1.1 that will boot up Xen 4.3.1 (Fedora 20)
> with IOMMU enabled. 
> 
> (XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0 flags 0xd7
> (XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x1 handle 0x8
> (XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0 flags 0xd7
> (XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x2 handle 0
> (XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0 flags 0
> (XEN) AMD-Vi: IVHD Special: 0000:00:00.1 variety 0x1 handle 0x8
> (XEN) IVHD Error: Conflicting IO-APIC 0x8 entries
> (XEN) AMD-Vi: IOMMU 0 Enabled.
> (XEN) AMD-Vi: Enabling global vector map
> (XEN) AMD-Vi: Using global interrupt remap table is not recommended (see
> XSA-36)!
> (XEN) I/O virtualisation enabled
> (XEN)  - Dom0 mode: Relaxed
> (XEN) Interrupt remapping enabled
> (XEN) ENABLING IO-APIC IRQs
> 
> My xen command line is:
> 
> dom0_mem=2048M,max:2048M dom0_max_vcpus=2 dom0_vcpus_pin loglvl=all
> guest_loglvl=all iommu=debug,verbose,no-amd-iommu-perdev-intremap
> 
> PCI passthrough of USB controllers works without issues, but I have not been
> able to passthrough a secondary graphics card (HD 5750) without getting a
> BSoD on my Win7 HVM.  I have had the same 5750 and VM image working on a
> GA-890FXA, but have not been able to get the combination working on the
> GA-990FXA.  If you are able to get IOMMU enabled, I will be interested to
> see if you get the graphics card to passthrough.
> 
> Good luck.
> 
> -Eric
> 
> 
> 
> 
> -----Original Message-----
> From: xen-users-bounces@xxxxxxxxxxxxx
> [mailto:xen-users-bounces@xxxxxxxxxxxxx] On Behalf Of Michaael
> Mair-Keimberger
> Sent: Sunday, December 22, 2013 7:08 AM
> To: xen-users@xxxxxxxxxxxxx
> Subject: [Xen-users] vga passthrogh on mainboards with broken ivrs tables
> 
> Hi Lists,
> 
> I'm having troubles getting vga passthrough working again with latest
> xen-4.3.1.
> 
> My system is a bulldozer amd 8core cpu with a gigabyte GA-990FXA-UD5
> motherboard.
> The host system uses a amd hd6850 while the guest (windows 7) uses a amd hd
> 7970.
> 
> So far everything worked great until xsa-36 [1]. I've haven't updated xen
> since this "fix", but yesterday i'll gave it another try. I've updated to
> xen-4.3.1 and tried to fix those broken ivrs tables with the newly
> introduced ivrx_ioapic kernel parameter. I'm not sure if this should help
> me, but at least it sounds promising.
> However i didn't get it to work and i hope someone can give my some hints
> how to fix that, otherwise i have to downgrade xen again.
> 
> 
> grub2 config:
> 
> menuentry 'Gentoo Linux XEN' {
>         root=hd0,1
>         multiboot /boot/xen.gz dom0_mem=8192M,max:8192M iommu=1
> dom0_max_vcpus=4 dom0_vcpus_pin ivrs_ioapic[8]=00:14.0
>         module /boot/gentoo-3.12.6 root=/dev/md3 radeon.pcie_gen2=1
> radeon.dpm=1 softlevel=xen xen-pciback.permissive
> xen-pciback.hide=(01:00.0)(01:00.1)(00:12.0)(00:12.2)
> }
> 
> lspci:
> 
> 00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI
> bridge (external gfx0 port B) (rev 02)
> 00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory
> Management Unit (IOMMU)
> 00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI
> bridge (PCI express gpp port B)
> 00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI
> bridge (PCI express gpp port H)
> 00:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI
> bridge (external gfx1 port A)
> 00:0d.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI
> bridge (external gfx1 port B)
> 00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
> 00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> 00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB EHCI Controller
> 00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> 00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB EHCI Controller
> 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller
> (rev 42)
> 00:14.1 IDE interface: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 IDE Controller (rev 40)
> 00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia
> (Intel HDA) (rev 40)
> 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0
> LPC host controller (rev 40)
> 00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI
> Bridge (rev 40)
> 00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
> 00:15.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900
> PCI to PCI bridge (PCIE port 0)
> 00:15.1 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900
> PCI to PCI bridge (PCIE port 1)
> 00:15.2 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI
> bridge (PCIE port 2)
> 00:15.3 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI
> bridge (PCIE port 3)
> 00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> 00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB EHCI Controller
> 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
> Function 0
> 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
> Function 1
> 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
> Function 2
> 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
> Function 3
> 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
> Function 4
> 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor
> Function 5
> 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI]
> Tahiti XT [Radeon HD 7970]
> 01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT HDMI
> Audio [Radeon HD 7970 Series]
> 02:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller
> (rev 01)
> 03:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s
> Controller (rev 11)
> 04:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI]
> Barts PRO [Radeon HD 6850]
> 04:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Barts HDMI
> Audio [Radeon HD 6800 Series]
> 05:0e.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)]
> IEEE 1394 OHCI Controller (rev c0)
> 06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
> RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
> 07:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller
> (rev 01)
> 08:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s
> Controller (rev 11)
> 
> lspci -t:
> 
> -[0000:00]-+-00.0
>            +-00.2
>            +-02.0-[01]--+-00.0
>            |            \-00.1
>            +-09.0-[02]----00.0
>            +-0a.0-[03]----00.0
>            +-0d.0-[04]--+-00.0
>            |            \-00.1
>            +-11.0
>            +-12.0
>            +-12.2
>            +-13.0
>            +-13.2
>            +-14.0
>            +-14.1
>            +-14.2
>            +-14.3
>            +-14.4-[05]----0e.0
>            +-14.5
>            +-15.0-[06]----00.0
>            +-15.1-[07]----00.0
>            +-15.2-[08]----00.0
>            +-15.3-[09]--
>            +-16.0
>            +-16.2
>            +-18.0
>            +-18.1
>            +-18.2
>            +-18.3
>            +-18.4
>            \-18.5
> 
> xl dmesg (shorten):
> 
> (XEN) Xen version 4.3.1 (@xonet.at) (x86_64-pc-linux-gnu-gcc (Gentoo 4.8.2
> p1.0, pie-0.5.8) 4.8.2) debug=n Sat Dec 21 11:55:39 CET 2013
> (XEN) Latest ChangeSet: 
> (XEN) Bootloader: GRUB 2.02~beta1
> (XEN) Command line: dom0_mem=8192M,max:8192M iommu=1 dom0_max_vcpus=4
> dom0_vcpus_pin ivrs_ioapic[8]=00:14.0 ...
> (XEN) IOAPIC[0]: apic_id 8, version 33, address 0xfec00000, GSI 0-23
> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 4026.984 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) xstate_init: using cntxt_size: 0x3c0 and states: 0x4000000000000007
> (XEN) IVHD Error: Conflicting IO-APIC 0x8 entries
> (XEN) AMD-Vi: Error initialization
> (XEN) I/O virtualisation disabled
> (XEN) ENABLING IO-APIC IRQs
> (XEN)  -> Using new ACK method
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Allocated console ring of 16 KiB.
> (XEN) HVM: ASIDs enabled.
> (XEN) SVM: Supported advanced features:
> (XEN)  - Nested Page Tables (NPT)
> (XEN)  - Last Branch Record (LBR) Virtualisation
> (XEN)  - Next-RIP Saved on #VMEXIT
> (XEN)  - VMCB Clean Bits
> (XEN)  - DecodeAssists
> (XEN)  - Pause-Intercept Filter
> (XEN)  - TSC Rate MSR
> (XEN) HVM: SVM enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) detected
> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> (XEN) spurious 8259A interrupt: IRQ7.
> 
> 
> I've tried a few other configuration's (like adding the ivrs_ioapic
> parameter to the module line in grub or also adding
> vfio_iommu_type1.allow_unsafe_interrupts=1
> to the cmdline), but so far i couldn't get it to work.
> 
> Since i'm not completely sure what i actually have to insert at the kernel
> parameter i'm glad if someone could give me some insight. However, since the
> output of xl dmesg is always the same, even though with or without the
> parameter, i might missed some additional parameters.
> 
> If you need any more information, I'll be glad to provide them.
> 
> Thanks in advance.
> 
> Michael
> 
> 
> 
> 
> 
> 
> 
> 
> [1] http://lists.xen.org/archives/html/xen-announce/2013-02/msg00002.html
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxx
> http://lists.xen.org/xen-users
> 

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users