RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

[Tim Moore <timothy.moore@xxxxxxxxxxx>]

I agree, if any I was expecting the Nvidia card to work the most ..

One interesting thing is that there is a fan on the Nvidia that spins down when 
the bios is loaded, with Windows it is evident when the bios is reset/loaded - 
just before the GUI is displayed.
When booting Xen the fan stays high / fast as the pciback is loaded to prevent 
Dom0 taking ownership.. boot the DomU (WinXP) and the fan spins down the same 
as booting a native Win32 host .. aka I assume Bios/Card is loaded/Reset here ..

Pls can you explain FLR's ?


-----Original Message-----
From: Jean Guyader [mailto:jean.guyader@xxxxxxxxx] 
Sent: 30 March 2009 15:57
To: Tim Moore
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough 
to <any> DomU

Hi Tim,

In the native case there is only one mapping, there isn't any
remapping ones the bios has been executed. It is different in the hvm
scenario.

Anyway, the nvidia graphic card should works fine as a secondary, side
by side with the cirrus. I don't know if this card support FLR's but
that would be good to have a look.

Jean

2009/3/30 Tim Moore <timothy.moore@xxxxxxxxxxx>:
>
> Hi Jean,
>
> I can understand the issue with the graphics card drivers, and I can see how 
> that could be able problem. I am using a very modern nvidia and ati card for 
> testing, Alex is also using a couple of other different varieties so we are 
> able to test with a few different scenarios...
>
> How would this work under an MS kernel? these manufacturers do support dual 
> cards and I have also tested my ATI/Nvidia host with a Windows OS and both 
> cards are loaded and work fine. This says to me that the drivers can deal 
> with PCI remapping of some sorts?
>
> Cheers,
> Tim
>
> -----Original Message-----
> From: Jean Guyader [mailto:jean.guyader@xxxxxxxxx]
> Sent: 30 March 2009 15:38
> To: Tim Moore
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough 
> to <any> DomU
>
> Hi Tim,
>
> Most of the time the problam doesn't come from the VT-d code but from
> the graphic card device drivers. Most of them doesn't really like PCI
> Bar remapping and try to use the old one.
> For instance the intel igfx has never been designed to be used as a
> secondary display adapter or even with another Host bridge than the
> one on the motherboard.
>
> Jean
>
> 2009/3/30 Tim Moore <timothy.moore@xxxxxxxxxxx>:
>> Hi Alex,
>>
>>
>>
>> When you say you had success with the onboard igfx, how much success? Have
>> you ever had a passthrough VGA display from a DomU ?
>>
>>
>>
>> I have also seen the write-up too although a lot of work has been done on
>> the VT-d support lately and I`m hoping to raise this again on the developer
>> list and start to document some of the problems/issues so as we can find a
>> way do solve it !
>>
>>
>>
>> Thanks for the reply! It still leaves this issue unsolved though ;(
>>
>>
>>
>> Tim
>>
>>
>>
>> ________________________________
>>
>> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
>>
>>         <any> DomU
>>
>> From: Alexia Benington <alexbenington@xxxxxxxxx>
>>
>> To: Tim Moore <timothy.moore@xxxxxxxxxxx>
>>
>> Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
>>
>>
>>
>> Hi,
>>
>>
>>
>> I've been trying to get this to work for some months, albeit not
>>
>> full-time. I concur that many would like to see it working. Some have
>>
>> posted that they managed to get it to work correctly, however, they
>>
>> almost always never reply to queries on how they got it to work, or
>>
>> their observations.
>>
>>
>>
>> I did manage to get rid of the yellow exclamation mark. You'll need to
>>
>> disable the emulated VGA bios, pretty much as described here
>>
>> http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>>
>>
>>
>> Even after doing so, some of the registers are not directly passed
>>
>> through. See http://lists.xensource.com/archives/html/xen-devel/2009-02/msg=
>>
>> 01131.html.
>>
>> I'm not sure if that may affect anything.
>>
>>
>>
>> I've experimented with an Intel DQ45CB, nVidia Quadro NVS 290, nVidia
>>
>> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>>
>> the onboard igfx as the primary, which is also being passed through.
>>
>>
>>
>> There is also a writeup which you might want to take a look
>>
>> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>>
>>
>>
>> I'm also trying to get DomU's output to the display.
>>
>>
>>
>> -Alex
>>
>>
>>
>> 2009/3/30 Tim Moore <timothy.moore@xxxxxxxxxxx>:
>>
>>> Hi
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have been testing xen-unstable.hg (29/03/09) over the past few days ..
>>
>>> trying to enable passthrough for VGA.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I am using Debian 5.0 (lenny) x64 and compiling from source each time. Tr=
>>
>> ied
>>
>>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>>
>>>
>>
>>>
>>
>>>
>>
>>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS P6T
>>
>>> Deluxe) with Core i7 920 CPU.
>>
>>>
>>
>>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388) and
>>
>>> remove the DMAR bail because of bad RMRR tables.
>>
>>>
>>
>>> I also have ATI X300 PCIe for Dom0 Console.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Using Xen parameter: iommu=3Dpassthrough,pv
>>
>>>
>>
>>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>>
>>>
>>
>>>
>>
>>>
>>
>>> I compiled pciback as embedded in the kernel and I have tried many differ=
>>
>> ent
>>
>>> scenarios:
>>
>>>
>>
>>>
>>
>>>
>>
>>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param) (no ATI card
>>
>>> installed)
>>
>>>
>>
>>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel param)
>>
>>>
>>
>>> 3) all scenarios with pciback as module and using echo devn >
>>
>>> /sys/bus/pci/drivers/pciback/new_slot and bind
>>
>>>
>>
>>>
>>
>>>
>>
>>> In all scenarios I can successfully see the correct PCI device in the
>>
>>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>>
>>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
>>
>>> (although I didn=92t load a driver) the lspci =96vvv showed the devices.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Now, I progressed to installing drivers under the Win32 DomU against the
>>
>>> Nvidia card, windows correctly identified the device and installed the
>>
>>> driver! (latest from nvidia.com). As we all know =96 Windows wanted a reb=
>>
>> oot-
>>
>>> beforehand a quick check of the device and it has a Yellow exclamation
>>
>>> indicating reboot required and resources unavailable.
>>
>>>
>>
>>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tried with the ATI X300 Primary (no NVidia connected) and went through th=
>>
>> e
>>
>>> same steps. Windows XP now boots without crash although the ATI device st=
>>
>> ill
>>
>>> shows a yellow exclamation and insufficient resources.
>>
>>>
>>
>>>
>>
>>>
>>
>>> To summarise, pciback functions as expected for PCIe in Dom0 and DomU see=
>>
>> s
>>
>>> the device. All tools work fine (xm pci-attach / pci-detach / pci-list) a=
>>
>> nd
>>
>>> devid show and are managed correctly.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read all that there is on this subject and have come to the
>>
>>> conclusion that this is a problem with qemu-dm and the memory mapping for
>>
>>> the new Video device, the Cirrus or stdvga card is always present which I
>>
>>> believe may be causing the problem. I have tried nographics=3D1 but qemu-=
>>
>> dm
>>
>>> still maps the Video BIOS.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Then I noticed that qemu-dm has the new options for =93-vga none=94 .. so=
>>
>>  I
>>
>>> built a wrapper script for qemu-dm to launch with this param. Unfortuntat=
>>
>> ely
>>
>>> the DomU doesn=92t startup and the qemu log does not show any mention of =
>>
>> the
>>
>>> Video memory ! so I believe the switch works, but the DomU then fails to
>>
>>> boot L
>>
>>>
>>
>>>
>>
>>>
>>
>>> To solve this and make it usable I think some development of qemu with
>>
>>> regards to the Cirrus VGA and memory management when using more than 1 vg=
>>
>> a
>>
>>> controller is necessary in order to remap the Cirrus or secondary vga
>>
>>> adapters in memory correctly. (its also difficult to configure a Windows
>>
>>> machine for a second vga when the primary is disabled lol.)
>>
>>>
>>
>>>
>>
>>>
>>
>>> Also, I have run strace using a qemu-dm wrapper (very verbose) and analys=
>>
>> ed
>>
>>> the results =96nothing seems obvious, but i`m not an expert by far =85.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Windows has been able to support multiple VGA cards for a long time; I
>>
>>> assume that MS is remapping these somehow?
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read in 2 different places that this has worked for others before
>>
>>> (without the legacy patch for igfx) and with xen-unstable.hg .. why does =
>>
>> it
>>
>>> not work for me ?
>>
>>>
>>
>>>
>>
>>>
>>
>>> Any help / direction to further debug would be helpful .. I think many
>>
>>> people would like to get this one working once and for all!
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tim
>>
>>>
>>
>>> _______________________________________________
>>
>>> Xen-devel mailing list
>>
>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>
>>> http://lists.xensource.com/xen-devel
>>
>>>
>>
>>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel
>>
>>
>
>
>
> --
> Jean Guyader
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
>



-- 
Jean Guyader

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel