Re: [Xen-users] Debian Xen pciback missing?

[Tobias Geiger <tobias.geiger@xxxxxxxxx>]

Hi Casey,

looks like nvidia brings with it not only a GPU and a HDMI pci-device, but 
also "NF200 PCIe 2.0 switch" ; Maybe this is one of the many reasons why 
NVIDIA GPUs make so much problems when passing them through in xen.

Sorry, my short answer is: get a ATI Card :)
With ATI it's - generally - no problem to pass it through in xen. no such PCI-
Device-ID/BUS plethora, no xen code patches to make - it "just works", at 
least here with a Radeon 6970. 

You may test David Techer's way of adapting xen to your individual nvidia 
setup, search (also in xen-devel) for his posts and urls to his blog.

Greetings and good luck anyways!
Tobias

Am Mittwoch, 7. März 2012, 01:55:05 schrieb Casey DeLorme:
> Hello again, and many thanks.  I bring a status update!
> 
> I was able to compile the Linux Kernel 3.2.9 with pciback built-in instead
> of as a module.  I also installed Xen Hypervisor 4.1 via aptitude and was
> able to adjust settings in grub.cfg after.
> 
> Xen boots, and the grub line appears to work:
> xen-pciback.hide=(01:00.0)(02:00.0)(03:00.0)(03:00.1)
> 
> Now when I run xm pci-list-assignable-devices, I get:
> 0000:03:00.0
> 0000:03:00.1
> 0000:01:00.0
> 0000:02:00.0
> 0000:02:02.0
> 0000:02:03.0
> 
> My objective is to achieve USB & VGA passthrough, but my graphics card
> appears to be sitting behind numerous devices.
> 
> My DomU Configuration:
> 
> name="windows7"
> maxmem=6912
> memory=6912
> #maxvcpus=6
> vcpus=4
> on_poweroff="destroy"
> on_reboot="restart"
> on_crash="destroy"
> localtime=0
> builder="hvm"
> device_model="/usr/lib/xen-4.1/bin/qemu-dm"
> kernel="/usr/lib/xen-4.1/boot/hvmloader"
> boot="dc"
> disk=[
>     'phy:/dev/guests/win7,hda,w',
>     'file:/home/xen/win7.iso,hdc:cdrom,r'
>   ]
> vif=[ 'bridge=xenbr0, model=e1000' ]
> stdvga=1
> sdl=0
> vnc=1
> vncunused=1
> viridian=1
> pae=1
> usbdevice='tablet'
> 
> 
> I can run xm create -c windows7 without any problems.
> 
> When I add this to the configuration:
> 
> pci=[
>     '03:00.0',
>     '03:00.1'
>   ]
> 
> I get this error:
> Error: pci: to avoid potential security issue, 0000:03:00.0 is not allowed
> to be assigned to guest since it is behind PCIe switch that does not
> support or enable ACS.
> 
> So I tried this:
> 
> pci=[
>     '01:00.0'
>     '03:00.0',
>     '03:00.1'
>   ]
> 
> However, I then receive this error:
> Error: pci: PCI Backend and pci-stub don't own device 0000:01:00.0
> 
> Even though it's listed as an assignable device, so I don't know if I
> misread what the card is sitting behind, or if there is some override I
> need to add to my configuration.
> 
> I can pass other devices, such as my USB and onboard Audio without any
> problems, for example:
> 
> pci=[
>     '00:1a.0'
>   ]
> 
> If I try to attach the device after starting the DomU I get:
> 
> xm pci-attach 3 0000:01:00.0
> Error: pci: PCI Backend and pci-stub don't own device 0000:01:00.0
> Usage: xm pci-attach [-o|--options=<opt>] <Domain> <domain:bus:slot.func>
> [virtual slot]
> 
> Insert a new pass-through pci device.
> 
> 
> So, I have two questions.
> 
> #1:  Is there a way to override the first error, if it's a "potential
> security" risk, surely there must be an option to ignore it?
> 
> #2:  Any ideas as to why the 01:00.0 devices is listed as assignable yet
> not available?  Do I need to pass a step further up?
> 
> 
> lspci -tv output:
> 
> -[0000:00]-+-00.0  Intel Corporation 2nd Generation Core Processor Family
> DRAM Controller
>            +-01.0-[01-05]----00.0-[02-05]--+-00.0-[03]--+-00.0  nVidia
> Corporation GF104 [GeForce GTX 460]
> 
>            |                               |            \-00.1  nVidia
> 
> Corporation GF104 High Definition Audio Controller
> 
>            |                               +-02.0-[04]--
>            |                               \-03.0-[05]--
> 
>            +-02.0  Intel Corporation 2nd Generation Core Processor Family
> Integrated Graphics Controller
>            +-16.0  Intel Corporation 6 Series/C200 Series Chipset Family
> MEI Controller #1
>            +-1a.0  Intel Corporation 6 Series/C200 Series Chipset Family
> USB Enhanced Host Controller #2
>            +-1b.0  Intel Corporation 6 Series/C200 Series Chipset Family
> High Definition Audio Controller
>            +-1c.0-[06]--
>            +-1c.5-[07]----00.0  ASMedia Technology Inc. ASM1062 Serial ATA
> Controller
>            +-1c.6-[08]----00.0  ASMedia Technology Inc. ASM1042 SuperSpeed
> USB Host Controller
>            +-1c.7-[09-12]----00.0-[0a-12]--+-01.0-[0b]----00.0  Broadcom
> Corporation NetLink BCM57781 Gigabit Ethernet PCIe
> 
>            |                               +-04.0-[0c]----00.0  ASMedia
> 
> Technology Inc. ASM1062 Serial ATA Controller
> 
>            |                               +-05.0-[0d]----00.0  Broadcom
> 
> Corporation NetLink BCM57781 Gigabit Ethernet PCIe
> 
>            |                               +-06.0-[0e]----00.0  VIA
> 
> Technologies, Inc. VT6315 Series Firewire Controller
> 
>            |                               +-07.0-[0f-10]----00.0-[10]--
>            |                               +-08.0-[11]----00.0  ASMedia
> 
> Technology Inc. ASM1042 SuperSpeed USB Host Controller
> 
>            |                               \-09.0-[12]----00.0  ASMedia
> 
> Technology Inc. ASM1042 SuperSpeed USB Host Controller
>            +-1d.0  Intel Corporation 6 Series/C200 Series Chipset Family
> USB Enhanced Host Controller #1
>            +-1f.0  Intel Corporation Z68 Express Chipset Family LPC
> Controller
>            +-1f.2  Intel Corporation 6 Series/C200 Series Chipset Family 6
> port SATA AHCI Controller
>            \-1f.3  Intel Corporation 6 Series/C200 Series Chipset Family
> SMBus Controller
> 
> lspci:
> 
> 00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family
> DRAM Controller (rev 09)
> 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core
> Processor Family PCI Express Root Port (rev 09)
> 00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core
> Processor Family Integrated Graphics Controller (rev 09)
> 00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series
> Chipset Family MEI Controller #1 (rev 04)
> 00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset
> Family USB Enhanced Host Controller #2 (rev 05)
> 00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family
> High Definition Audio Controller (rev 05)
> 00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family
> PCI Express Root Port 1 (rev b5)
> 00:1c.5 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family
> PCI Express Root Port 6 (rev b5)
> 00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family
> PCI Express Root Port 7 (rev b5)
> 00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family
> PCI Express Root Port 8 (rev b5)
> 00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset
> Family USB Enhanced Host Controller #1 (rev 05)
> 00:1f.0 ISA bridge: Intel Corporation Z68 Express Chipset Family LPC
> Controller (rev 05)
> 00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset
> Family 6 port SATA AHCI Controller (rev 05)
> 00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus
> Controller (rev 05)
> 01:00.0 PCI bridge: nVidia Corporation NF200 PCIe 2.0 switch for mainboards
> (rev a3)
> 02:00.0 PCI bridge: nVidia Corporation NF200 PCIe 2.0 switch for mainboards
> (rev a3)
> 02:02.0 PCI bridge: nVidia Corporation NF200 PCIe 2.0 switch for mainboards
> (rev a3)
> 02:03.0 PCI bridge: nVidia Corporation NF200 PCIe 2.0 switch for mainboards
> (rev a3)
> 03:00.0 VGA compatible controller: nVidia Corporation GF104 [GeForce GTX
> 460] (rev a1)
> 03:00.1 Audio device: nVidia Corporation GF104 High Definition Audio
> Controller (rev a1)
> 07:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA
> Controller (rev 01)
> 08:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host
> Controller
> 09:00.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev ba)
> 0a:01.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev ba)
> 0a:04.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev ba)
> 0a:05.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev ba)
> 0a:06.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev ba)
> 0a:07.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev ba)
> 0a:08.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev ba)
> 0a:09.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev ba)
> 0b:00.0 Ethernet controller: Broadcom Corporation NetLink BCM57781 Gigabit
> Ethernet PCIe (rev 10)
> 0c:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA
> Controller (rev 01)
> 0d:00.0 Ethernet controller: Broadcom Corporation NetLink BCM57781 Gigabit
> Ethernet PCIe (rev 10)
> 0e:00.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6315 Series Firewire
> Controller (rev 01)
> 0f:00.0 PCI bridge: ASMedia Technology Inc. ASM108x PCIe to PCI Bridge
> Controller (rev 01)
> 11:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host
> Controller
> 12:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host
> Controller
> 
> On Tue, Mar 6, 2012 at 1:29 PM, Casey DeLorme <cdelorme@xxxxxxxxx> wrote:
> > Hello Tobias, and thanks for the suggestions!
> > 
> > I had been told that Debian Squeeze came with pciback "out of the box",
> > but clearly that was a misnomer (given the "m" in my kernel config), and
> > as mentioned in my initial email I tried adding lines to grub with no
> > results.
> > 
> > Using the find command I was able to find three records with pci:
> > /lib/modules/3.2.0-1-amd64/kernel/drivers/pci/xen-pcifront.ko
> > /lib/modules/3.2.0-1-amd64/kernel/drivers/xen/xen-pciback
> > /lib/modules/3.2.0-1-amd64/kernel/drivers/xen/xen-pciback/xen-pciback.ko
> > 
> > Also, I can type just "modprobe xen-pciback" without any errors. 
> > However, when I attempt to send a command to it, it fails:
> > sudo modprobe xen-pciback 'hide=(03:00.0)'
> > ERROR: could not insert 'xen_pciback': No such file or directory
> > 
> > I am new enough to Xen that this really doesn't give me enough to go on.
> > 
> >  It's trying to insert... someplace, and it's using xen_pciback (not
> > 
> > xen-pciback?)
> > 
> > 
> > Your other suggestion is much more appealing, but I want to ask a few
> > questions before I give that a spin.
> > 
> > Question #1:
> > I would be compiling the whole kernel, correct?  I just want to make sure
> > I'm not missing some faster or easier method that would append
> > functionality to an existing kernel.
> > 
> > Question #2:
> > With a custom Kernel will I be able to use the package manager to install
> > Xen, or will I have to compile Xen from source to have it recognize my
> > custom Linux Kernel?
> > 
> > Question #3:
> > If I wanted to compile Xen from source, or the package manager is not an
> > option, how can I compile the latest Xen source with my custom kernel?
> > 
> > I found instructions for 4.1.1 in a pdf, but they didn't work verbatim
> > with 4.1.2, and I ended up with 4x .gz files in my boot directory, and
> > half a dozen new grub records, none of which booted.
> > 
> > Compiled from source would be ideal as I could try fixing the efi-boot
> > memory issues, but without some kind of guide for 4.1.2 I would be lost
> > at accomplishing this.
> > 
> > ~Casey


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users