[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Re : Available PCIe lanes for VGA passthrough


  • To: Sandi Romih <romihs.forums@xxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
  • From: David TECHER <davidtecher@xxxxxxxx>
  • Date: Tue, 17 Jan 2012 15:38:48 +0000 (GMT)
  • Delivery-date: Tue, 17 Jan 2012 15:39:06 +0000
  • Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.fr; h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=d2QcRVsnZDpvAsXonWx2cgJ5fCkaEWOlD4bXr/qTvDBLWvZQFtAxT3ZZ5tgiDqc+9uvv7p7KigoeWCh40xhswbSTUVm9QBi9CPqalr7b0F+MLR2PIEU8rCVJXGF0h+Mu9coLYdCb32QAFeT4ZbvjjO534Afex73WUWioX6ciXd8=;
  • List-id: Xen developer discussion <xen-devel.lists.xensource.com>

Hi

First of all, be sure that you have tested the revision <= 24491.

I mean

hg clone -r $rev http://xenbits.xensource.com/staging/xen-unstable.hg/ xen-unstable.hg-rev-${rev}

rev<=24491

revision between, 24492 and 24516 may not work for the moment due to revision 24492 (ATS device driver for IOMMU+HVM) it may not work.

For the moment I did not try revision > 24516.

Then check that  tools/firmware/hvmloader/acpi/dsdt.asl is updated according to the 3 mem ranges.

dmesg | grep 03:00.0 | grep BAR

Have a look on my blog. There is an example Section 6. "Quick instructions to install Xen with patches" - Point 5


Finally rebuild Xen.

De : Sandi Romih <romihs.forums@xxxxxxxxx>
À : xen-devel@xxxxxxxxxxxxxxxxxxx
Envoyé le : Mardi 17 Janvier 2012 14h02
Objet : Re: [Xen-devel] Available PCIe lanes for VGA passthrough

Hello everyone,

Looks like my initial email to xen-devel did not get delivered or something...
Please see it below this email.

Okay, here is some info about my system:

lspci | grep -i vga
03:00.0 VGA compatible controller: nVidia Corporation GF100 [GeForce GTX 480] (rev a3)
06:04.0 VGA compatible controller: Matrox Graphics, Inc. MGA G200eW WPCM450 (rev 0a)


lspci | grep -i 03:00
03:00.0 VGA compatible controller: nVidia Corporation GF100 [GeForce GTX 480] (rev a3)
03:00.1 Audio device: nVidia Corporation GF100 High Definition Audio Controller (rev a1)


dmesg | grep -i 03:00
[    0.000000] Command line: placeholder root=UUID=d5f5207b-d2aa-4f19-b51d-bc2c727b9e8f ro nomodeset xen-pciback.passthrough=1 xen-pciback.permissive xen-pciback.hide=(03:00.0)(03:00.1) quiet
[    5.114443] Kernel command line: placeholder root=UUID=d5f5207b-d2aa-4f19-b51d-bc2c727b9e8f ro nomodeset xen-pciback.passthrough=1 xen-pciback.permissive xen-pciback.hide=(03:00.0)(03:00.1) quiet
[    5.356290] pci 0000:03:00.0: [10de:06c0] type 0 class 0x000300
[    5.356310] pci 0000:03:00.0: reg 10: [mem 0xf8000000-0xf9ffffff]
[    5.356332] pci 0000:03:00.0: reg 14: [mem 0xd8000000-0xdfffffff 64bit pref]
[    5.356353] pci 0000:03:00.0: reg 1c: [mem 0xd4000000-0xd7ffffff 64bit pref]
[    5.356368] pci 0000:03:00.0: reg 24: [io  0xdc00-0xdc7f]
[    5.356383] pci 0000:03:00.0: reg 30: [mem 0xfae80000-0xfaefffff pref]
[    5.356475] pci 0000:03:00.1: [10de:0be5] type 0 class 0x000403
[    5.356494] pci 0000:03:00.1: reg 10: [mem 0xfae7c000-0xfae7ffff]
[    5.425416] vgaarb: device added: PCI:0000:03:00.0,decodes=io+mem,owns=none,locks=none
[    5.425438] vgaarb: bridge control possible 0000:03:00.0
[    5.443721] pciback 0000:03:00.0: seizing device
[    5.443726] pciback 0000:03:00.1: seizing device
[    5.713088] pciback 0000:03:00.0: Signaling PME through PCIe PME interrupt
[    5.713090] pciback 0000:03:00.1: Signaling PME through PCIe PME interrupt
[    5.713566] pciback 0000:03:00.1: PCI INT B -> GSI 25 (level, low) -> IRQ 25
[    5.713574] pciback 0000:03:00.1: PCI INT B disabled
[    5.713610] pciback 0000:03:00.0: enabling device (0146 -> 0147)
[    5.713627] pciback 0000:03:00.0: PCI INT A -> GSI 26 (level, low) -> IRQ 26
[    5.713633] pciback 0000:03:00.0: PCI INT A disabled


xl pci-list-assignable-devices 
0000:03:00.0
0000:03:00.1


cat /etc/default/grub 
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet"
GRUB_CMDLINE_LINUX="nomodeset xen-pciback.passthrough=1 xen-pciback.permissive xen-pciback.hide=(03:00.0)(03:00.1)"
GRUB_DISABLE_OS_PROBER=true
GRUB_CMDLINE_XEN="dom0_mem=1G"


From all that, I assume that the VGA card, and its on-board audio device, is owned by pciback.


This is how I start the domain:
xl create xenwinxp.cfg 
Parsing config file xenwinxp.cfg
WARNING: ignoring "kernel" directive for HVM guest. Use "firmware_override" instead if you really want a non-default firmware
xc: info: VIRTUAL MEMORY ARRANGEMENT:
  Loader:        0000000000100000->00000000001819b4
  TOTAL:         0000000000000000->00000000bf800000
  ENTRY ADDRESS: 0000000000100000
xc: info: PHYSICAL MEMORY ALLOCATION:
  4KB PAGES: 0x0000000000000200
  2MB PAGES: 0x00000000000003fb
  1GB PAGES: 0x0000000000000001
libxl: error: libxl_pci.c:776:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:03:00.0
Daemon running with PID 3001

Quick confirmation that it is up and running:
xl list
Name                                        ID   Mem VCPUs State Time(s)
Domain-0                                     0  1024    12     r-----     255.7
winxp                                        1  3067     4     -b----      61.2

This is my xen config file:
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory = 3072
# shadow_memory = 1024
name = "winxp"
vcpus='4'
vif = [ 'bridge=eth0,mac=00:14:3e:00:8f:c2' ]
disk = ['file:/xen-vms/winxp/xenwinxp.img,hda,w','file:/xen-vms/isos/winxp.iso,hdc:cdrom,r']
boot="cd"
sdl=0
vnc=1
vnclisten="0.0.0.0"
vncconsole=1
vncpasswd=''
acpi=1
apic=1
xen_extended_power_mgmt=1
pae=1
arch='x86_64'
hpet = 1
hap = 1
viridian = 1
monitor=1
serial='pty'
# keymap='fr'
# soundhw = "all"
# audio="on"
ne2000=0
on_reboot   = 'restart'
on_crash    = 'restart'
xen_platform_pci=1
gfx_passthru=0
pci  = [ "03:00.0,msitranslate=1,power_mgmt=1" ]
acpi_s3 = 1
acpi_s4 = 1


The Windows XP boots up fine, I can see the GTX480 in the device manager, but it is not useable.

 Now, let me try gfx_passthru=1

Create the domain:
xl create xenwinxp.cfg 
Parsing config file xenwinxp.cfg
WARNING: ignoring "kernel" directive for HVM guest. Use "firmware_override" instead if you really want a non-default firmware
xc: info: VIRTUAL MEMORY ARRANGEMENT:
  Loader:        0000000000100000->00000000001819b4
  TOTAL:         0000000000000000->00000000bf800000
  ENTRY ADDRESS: 0000000000100000
xc: info: PHYSICAL MEMORY ALLOCATION:
  4KB PAGES: 0x0000000000000200
  2MB PAGES: 0x00000000000003fb
  1GB PAGES: 0x0000000000000001
libxl: error: libxl_pci.c:776:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:03:00.0
Daemon running with PID 3454

When the domain starts, my dom0 screen attached to the 06:04.0 VGA controller goes blank, and after a few seconds I get a message from my monitor that 'this video format is not supported'. I can get back to my dom0 by first going Ctrl-Alt-F1 (where I will see a whole bunch of garbage in the screen) and then Ctrl-Alt-F7.

The domain is running:
xl list
Name                                        ID   Mem VCPUs State Time(s)
Domain-0                                     0  1024    12     r-----     385.3
winxp                                        3  3063     1     r-----     324.5

And if I check out what is happening through vncviewer, I see just the QEMU console (QEMU 0.10.2 monitor - type 'help' for more information)

And finally, I see that the VGA card is no longer available:
xl pci-list-assignable-devices 
0000:03:00.1


I have checked all three of the video cards outputs, no video signal anywhere.


Any ideas about what could be causing the problems I am encountering?
I would be grateful for any info.

Thanks in advance.

Sandi





On Tue, Jan 17, 2012 at 11:10 AM, Sandi Romih <romihs.forums@xxxxxxxxx> wrote:
Hello,

I hope that I am posting the correct list with this question.

I have been trying to get VGA passthrough working on my system for some time now without success.
I do not have the system with me right now, so I will not be able to provide any logs or error messages right now. I will be able to to do so in a couple of hours.

I have a VT-d enabled system (Motherboard and CPU) on which I wish to pass the secondary graphics card (EVGA GTX480 SE) through to the HVM (WinXP 32-bit).
Currently I am testing with Debian Squeeze, using the 3.1.8 kernel and Xen 4.2.unstable according to these instructions: http://www.davidgis.fr/blog/index.php?2011/12/07/860-xen-42unstable-patches-for-vga-pass-through

I can only boot the WinXP client via the VNC console (gfx_passthrough=0), and from within WinXP I see that the OS has detected the GTX480 card (as its secondary card), but I am not able to use it. I have tried the 275.33 drives, and they make no difference.

My question is related to the number of PCIe lanes I have available to the graphics card.
I only have 8 lanes available for the GTX480 (which is a PCIe x16 card), could this pose an issue for Xen?


I plan to test VGA passthrough with an ATI graphics card (also in a PCIe x8 slot) later to see if it will work or not. I will update what I find.

Regards

Sandi


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.