Xen project Mailing List

Sandi

On Wed, Jan 18, 2012 at 11:59 AM, David TECHER <davidtecher@xxxxxxxx> wrote:

Hi Sandi,

To be honest, your mail is little confused.

I do not understand why you wrote "This confirms that linux uses the card fine with only 8 lanes of PCIe."

then you wrote

"Would Xen also be okay for this card with only 8 lanes available?"

My qyestion is: everything is Ok now? Let me know.

Thanks to CC xen-user too to your reply if it can help someones :)

Thanks.

Regards.

David
De : Sandi Romih <romihs.forums@xxxxxxxxx>

À : David TECHER <davidtecher@xxxxxxxx>
Envoyé le : Mardi 17 Janvier 2012 22h42
Objet : Re: Re : [Xen-devel] Available PCIe lanes for VGA passthrough
Hello,

I tried running 'dmesg | grep 03:00.0 | grep BAR', but I cant because the video card details in dmesg do not have any BAR parameters assigned to them.

I can only run 'dmesg | grep 03:00.0 | grep mem', and I get this:

[ 1.469650] pci 0000:03:00.0: reg 10: [mem 0xf8000000-0xf9ffffff]
[ 1.469660] pci 0000:03:00.0: reg 14: [mem 0xd8000000-0xdfffffff 64bit pref]
[ 1.469669] pci 0000:03:00.0: reg 1c: [mem 0xd4000000-0xd7ffffff 64bit pref]

[ 1.469683] pci 0000:03:00.0: reg 30: [mem 0xfae80000-0xfaefffff pref]

I do not have the nVidia drivers installed in Debian, could that be the reason why the BARs are not set?

Can I still just use these values to change dsdt.asl?
This is what I have done anyway.

I have installed the driver for this card now, and it seams to function properly, at least I can achieve the full resolution of the display. This confirms that linux uses the card fine with only 8 lanes of PCIe.

Would Xen also be okay using this card with only 8 lanes available?

Thanks for the help.

Sandi
On Tue, Jan 17, 2012 at 4:38 PM, David TECHER <davidtecher@xxxxxxxx> wrote:
Hi

First of all, be sure that you have tested the revision <= 24491.

I mean
hg clone -r $rev http://xenbits.xensource.com/staging/xen-unstable.hg/ xen-unstable.hg-rev-${rev}




rev<=24491
revision between, 24492 and 24516 may not work for the moment due to revision 24492 (ATS device driver for IOMMU+HVM) it may not work.

For the moment I did not try revision > 24516.

Then check that tools/firmware/hvmloader/acpi/dsdt.asl is updated according to the 3 mem ranges.
dmesg | grep 03:00.0 | grep BAR

Have a look on my blog. There is an example Section 6. "Quick instructions to install Xen with patches" - Point 5
Finally rebuild Xen.

De : Sandi Romih <romihs.forums@xxxxxxxxx>
À : xen-devel@xxxxxxxxxxxxxxxxxxx
Envoyé le : Mardi 17 Janvier 2012 14h02
Objet : Re: [Xen-devel] Available PCIe lanes for VGA passthrough

Hello everyone,

Looks like my initial email to xen-devel did not get delivered or something...
Please see it below this email.

Okay, here is some info about my system:

lspci | grep -i vga
03:00.0 VGA compatible controller: nVidia Corporation GF100 [GeForce GTX 480] (rev a3)
06:04.0 VGA compatible controller: Matrox Graphics, Inc. MGA G200eW WPCM450 (rev 0a)

lspci | grep -i 03:00
03:00.0 VGA compatible controller: nVidia Corporation GF100 [GeForce GTX 480] (rev a3)
03:00.1 Audio device: nVidia Corporation GF100 High Definition Audio Controller (rev a1)

dmesg | grep -i 03:00
[ 0.000000] Command line: placeholder root=UUID=d5f5207b-d2aa-4f19-b51d-bc2c727b9e8f ro nomodeset xen-pciback.passthrough=1 xen-pciback.permissive xen-pciback.hide=(03:00.0)(03:00.1) quiet

[ 5.114443] Kernel command line: placeholder root=UUID=d5f5207b-d2aa-4f19-b51d-bc2c727b9e8f ro nomodeset xen-pciback.passthrough=1 xen-pciback.permissive xen-pciback.hide=(03:00.0)(03:00.1) quiet
[ 5.356290] pci 0000:03:00.0: [10de:06c0] type 0 class 0x000300

[ 5.356310] pci 0000:03:00.0: reg 10: [mem 0xf8000000-0xf9ffffff]
[ 5.356332] pci 0000:03:00.0: reg 14: [mem 0xd8000000-0xdfffffff 64bit pref]
[ 5.356353] pci 0000:03:00.0: reg 1c: [mem 0xd4000000-0xd7ffffff 64bit pref]

[ 5.356368] pci 0000:03:00.0: reg 24: [io 0xdc00-0xdc7f]
[ 5.356383] pci 0000:03:00.0: reg 30: [mem 0xfae80000-0xfaefffff pref]
[ 5.356475] pci 0000:03:00.1: [10de:0be5] type 0 class 0x000403

[ 5.356494] pci 0000:03:00.1: reg 10: [mem 0xfae7c000-0xfae7ffff]
[ 5.425416] vgaarb: device added: PCI:0000:03:00.0,decodes=io+mem,owns=none,locks=none
[ 5.425438] vgaarb: bridge control possible 0000:03:00.0

[ 5.443721] pciback 0000:03:00.0: seizing device
[ 5.443726] pciback 0000:03:00.1: seizing device
[ 5.713088] pciback 0000:03:00.0: Signaling PME through PCIe PME interrupt
[ 5.713090] pciback 0000:03:00.1: Signaling PME through PCIe PME interrupt

[ 5.713566] pciback 0000:03:00.1: PCI INT B -> GSI 25 (level, low) -> IRQ 25
[ 5.713574] pciback 0000:03:00.1: PCI INT B disabled
[ 5.713610] pciback 0000:03:00.0: enabling device (0146 -> 0147)

[ 5.713627] pciback 0000:03:00.0: PCI INT A -> GSI 26 (level, low) -> IRQ 26
[ 5.713633] pciback 0000:03:00.0: PCI INT A disabled

xl pci-list-assignable-devices

0000:03:00.0
0000:03:00.1

cat /etc/default/grub
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`

GRUB_CMDLINE_LINUX_DEFAULT="quiet"
GRUB_CMDLINE_LINUX="nomodeset xen-pciback.passthrough=1 xen-pciback.permissive xen-pciback.hide=(03:00.0)(03:00.1)"
GRUB_DISABLE_OS_PROBER=true

GRUB_CMDLINE_XEN="dom0_mem=1G"

From all that, I assume that the VGA card, and its on-board audio device, is owned by pciback.

This is how I start the domain:
xl create xenwinxp.cfg
Parsing config file xenwinxp.cfg
WARNING: ignoring "kernel" directive for HVM guest. Use "firmware_override" instead if you really want a non-default firmware

xc: info: VIRTUAL MEMORY ARRANGEMENT:
Loader: 0000000000100000->00000000001819b4
TOTAL: 0000000000000000->00000000bf800000
ENTRY ADDRESS: 0000000000100000

xc: info: PHYSICAL MEMORY ALLOCATION:
4KB PAGES: 0x0000000000000200
2MB PAGES: 0x00000000000003fb
1GB PAGES: 0x0000000000000001
libxl: error: libxl_pci.c:776:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:03:00.0

Daemon running with PID 3001

Quick confirmation that it is up and running:
xl list
Name ID Mem VCPUs State Time(s)

Domain-0 0 1024 12 r----- 255.7
winxp 1 3067 4 -b---- 61.2

This is my xen config file:

kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory = 3072
# shadow_memory = 1024
name = "winxp"
vcpus='4'
vif = [ 'bridge=eth0,mac=00:14:3e:00:8f:c2' ]
disk = ['file:/xen-vms/winxp/xenwinxp.img,hda,w','file:/xen-vms/isos/winxp.iso,hdc:cdrom,r']
boot="cd"
sdl=0

vnc=1
vnclisten="0.0.0.0"
vncconsole=1
vncpasswd=''
acpi=1
apic=1
xen_extended_power_mgmt=1
pae=1
arch='x86_64'

hpet = 1
hap = 1
viridian = 1
monitor=1
serial='pty'
# keymap='fr'
# soundhw = "all"
# audio="on"
ne2000=0

>
on_reboot = 'restart'
on_crash = 'restart'
xen_platform_pci=1
gfx_passthru=0
pci = [ "03:00.0,msitranslate=1,power_mgmt=1" ]

acpi_s3 = 1
acpi_s4 = 1

The Windows XP boots up fine, I can see the GTX480 in the device manager, but it is not useable.

Now, let me try gfx_passthru=1

Create the domain:
xl create xenwinxp.cfg
Parsing config file xenwinxp.cfg
WARNING: ignoring "kernel" directive for HVM guest. Use "firmware_override" instead if you really want a non-default firmware

xc: info: VIRTUAL MEMORY ARRANGEMENT:
Loader: 0000000000100000->00000000001819b4
TOTAL: 0000000000000000->00000000bf800000
ENTRY ADDRESS: 0000000000100000

xc: info: PHYSICAL MEMORY ALLOCATION:
4KB PAGES: 0x0000000000000200
2MB PAGES: 0x00000000000003fb
1GB PAGES: 0x0000000000000001
libxl: error: libxl_pci.c:776:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:03:00.0

Daemon running with PID 3454

When the domain starts, my dom0 screen attached to the 06:04.0 VGA controller goes blank, and after a few seconds I get a message from my monitor that 'this video format is not supported'. I can get back to my dom0 by first going Ctrl-Alt-F1 (where I will see a whole bunch of garbage in the screen) and then Ctrl-Alt-F7.

The domain is running:
xl list
Name ID Mem VCPUs State Time(s)

Domain-0 0 1024 12 r----- 385.3
winxp 3 3063 1 r----- 324.5

And if I check out what is happening through vncviewer, I see just the QEMU console (QEMU 0.10.2 monitor - type 'help' for more information)

And finally, I see that the VGA card is no longer available:
xl pci-list-assignable-devices
0000:03:00.1

I have checked all three of the video cards outputs, no video signal anywhere.

Any ideas about what could be causing the problems I am encountering?
I would be grateful for any info.

Thanks in advance.

Sandi

On Tue, Jan 17, 2012 at 11:10 AM, Sandi Romih <romihs.forums@xxxxxxxxx> wrote:

Hello,

I hope that I am posting the correct list with this question.

I have been trying to get VGA passthrough working on my system for some time now without success.

I do not have the system with me right now, so I will not be able to provide any logs or error messages right now. I will be able to to do so in a couple of hours.

I have a VT-d enabled system (Motherboard and CPU) on which I wish to pass the secondary graphics card (EVGA GTX480 SE) through to the HVM (WinXP 32-bit).
Currently I am testing with Debian Squeeze, using the 3.1.8 kernel and Xen 4.2.unstable according to these instructions: http://www.davidgis.fr/blog/index.php?2011/12/07/860-xen-42unstable-patches-for-vga-pass-through

I can only boot the WinXP client via the VNC console (gfx_passthrough=0), and from within WinXP I see that the OS has detected the GTX480 card (as its secondary card), but I am not able to use it. I have tried the 275.33 drives, and they make no difference.

My question is related to the number of PCIe lanes I have available to the graphics card.

I only have 8 lanes available for the GTX480 (which is a PCIe x16 card), could this pose an issue for Xen?

I plan to test VGA passthrough with an ATI graphics card (also in a PCIe x8 slot) later to see if it will work or not. I will update what I find.

Regards

Sandi

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

Re: [Xen-users] Re : Re : [Xen-devel] Available PCIe lanes for VGA passthrough