Re: [Xen-devel] [Xen-users] HVM PCI Passthrough: Code 12: Undersized PCI MMIO region?

[Pasi Kärkkäinen <pasik@xxxxxx>]

On Tue, Dec 02, 2014 at 10:08:35PM -0500, Stephen Oberholtzer wrote:
> All,
> 

Hello,

> Please bear with me.  I don't fully understand all of the mechanisms
> involved here, so this message contains a large quantity of data,
> including some links to even larger quantities of data.
> The short version is: I think something isn't reserving enough (GPFN)
> address space for MMIO.
>

This issue probably should be taken to xen-devel mailinglist.. CC added..


> I'm trying to do a three-part PCI passthrough to a HVM domU running Windows 
> 8.1:
> 00:14.0, An xHCI (USB 3.0) controller
> 01:00.0, Function 0 of an ATI card (VGA controller, non-primary passthrough)
> 01:00.1, Function 1 of the same ATI card (HDMI audio)
> 
> The USB controller seems to work fine.  USB mouse/keyboard are
> functional, and I plugged in a few Mass Storage devices and they work
> fine.
> The HDMI audio controller is harder to vet, because I don't have
> anything that will process audio without an associated video stream.
> However, I don't get any complains from the software.
> 
> For the the VGA controller (01:00.0), Windows reports "Code 12" and
> complains about insufficient resources being available for the device.
> 
> I tried to gather all of the information I could on the memory layouts
> involved, from three sources:
> (1) From dom0 and Xen itself
> (2) From domU running Windows, using a program called "SIV" (System
> Information Viewer)
> (3) From domU running Linux (by booting a Knoppix live DVD image)
> 
> ---
> 
> Information from dom0:
> http://www.klozoff.org/xen/dom0-xl-info.txt - 'xl info'
> http://www.klozoff.org/xen/dom0-xl-dmesg.txt - 'xl dmesg'
> http://www.klozoff.org/xen/dom0-proc_iomem.txt - 'cat /proc/iomem'
> http://www.klozoff.org/xen/dom0-lspci-output.txt - 'lspci -vvv' for
> the three devices I'm forwarding.
> 
> Information from domU (Windows):
> http://www.klozoff.org/xen/domU-device-manager-info.txt - Text
> transcribed from Windows' "Device Manager"
> http://www.klozoff.org/xen/domU-siv-usb-controller-info.txt -
> Information on 00:14.0 from SIV
> http://www.klozoff.org/xen/domU-siv-video-card-info.txt - Information
> on 01:00.0 from SIV
> http://www.klozoff.org/xen/domU-siv-hdmi-audio-info.txt - Information
> on 01:00.1 from SIV
> 
> Information from domU (Linux/Knoppix):
> http://www.klozoff.org/xen/domU-knoppix-proc-iomem.txt - 'cat /proc/iomem'
> http://www.klozoff.org/xen/domU-knoppix-lspci-output.txt - 'lspci
> -vvv' for the three devices.
> 
> 
> ---
> 
> Of particular interest to me is domU-knoppix-lspci-output.txt, which
> for the VGA controller, says:
> 
>     Region 0: Memory at <ignored> (64-bit, prefetchable)
>     Region 2: Memory at f3040000 (64-bit, non-prefetchable) [size=256K]
> 
> In dom0, this showed as:
> 
>     Region 0: Memory at d0000000 (64-bit, prefetchable) [size=256M]
>     Region 2: Memory at efc00000 (64-bit, non-prefetchable) [size=256K]
> 
> 
> I looked closer by running the following script:
> 
> perl -lne '/^([0-9a-f]+)-([0-9a-f]+)/ && print $_, " (", ((hex($2) -
> hex($1)) >> 20), " MB)"' /proc/iomem
> 
> In dom0, I see this:
> 
> 00100000-99f53fff : System RAM (2462 MB)
> bf200000-feafffff : PCI Bus 0000:00 (1016 MB)   <- Region 0 is in this range
> 100000000-83edfffff : System RAM (29677 MB)
> 
> 
> In domU, I see this:
> 00100000-effecfff : System RAM (3838 MB)
> f0000000-fbffffff : PCI Bus 0000:00 (191 MB)    <- Definitely not big
> enough to hold the 256MB we need
> 100000000-20f7fffff : System RAM (4343 MB)
> 
> Now, I don't fully understand how this stuff all works, but here's
> what appears to be happening: Xen isn't reserving enough contiguous
> GPFN (virtual machine address space) for all of the devices on the
> virtual PCI bus -- in particular, for what is probably a big honkin'
> window into the card's onboard RAM.
> 
> So my two questions are:
> (1) Is my guess correct?  Or at least close?
> (2) What the heck do I do about it? I don't see any options in
> xl.cfg(5) for specifying the size of the MMIO region for the virtual
> PCI bus, or anything similar to that.
> 
> 
> NOTE: If I assign 2048MB or less RAM to this VM, the VM crashes at
> bootup, and has a nasty tendency to take the host with it (lockup).
> 
> -- 
> -- Stevie-O
> Real programmers use COPY CON PROGRAM.EXE
> 


-- Pasi


> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxx
> http://lists.xen.org/xen-users

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel