Re: [Xen-devel] q35 in xen? vfio in xen?

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

On Mon, Feb 24, 2014 at 07:28:32PM +0000, Zhang, Eniac wrote:
> Hi Konrad,
> 
> Thanks for the info.  Your guest sees the virtual function as  pci device 
> just like I had suspected.  Unfortunately that won't work for me.  I guess I 
> have to take a hard look at implement pci-e passthrough using pciback then.

You won't have to do it with pciback. Keep in mind that pciback just "holds"
the device so that other drivers (like ixbgvf) don't use it.

'xl' ends up doing the proper hypercall to assign the device to
the guest. And QEMU also does it set of calls to setup the
BARS, interrupts, deal with MSI-X, etc.

What you are going to have to look at is QEMU - and how to make it
work with the newer emulated chipset.

Stefano (CC-ed) here is the maintainer of QEMU Xen pieces. Anthony
(CC-ed as well) had backported the proper pieces in QEMU to do
PCI passthrough.

Looking forward to your patches and we will be more than happy
to help you upstream them!

> 
> Regards/Eniac
> 
> -----Original Message-----
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@xxxxxxxxxx] 
> Sent: Monday, February 24, 2014 9:40 AM
> To: Zhang, Eniac
> Cc: xen-devel@xxxxxxxxxxxxx
> Subject: Re: [Xen-devel] q35 in xen? vfio in xen?
> 
> On Sat, Feb 22, 2014 at 08:06:50AM +0000, Zhang, Eniac wrote:
> > Hi Konrad,
> > 
> > Here's what I see when start a VM under xen using pciback to pass a pci-e 
> > device into domU.  The device can be seen by guest, and also functioning 
> > fine.  But it's not seen as a pci-e device, rather, it looks just like an 
> > ordinary pci device because only the first 0x100 bytes of its configuration 
> > space is accessible.  So if a driver needs to use data in the extended 
> > configuration space for certain features, it will fail.
> > 
> > When you say you "did PCIe pass through of an VF of an SR-IOV device".  Are 
> > you actually using it as a pci-e device or have throttled it back to pci 
> > mode without aware of the difference?  If you did see the pci-e device in 
> > guest, can you share your xl.cfg file as well as lspci/lspci -t/lspci -xxxx 
> > output from guest?
> 
> # lspci
> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> 00:02.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
> 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> 00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 
> 01) # lspci -t
> -[0000:00]-+-00.0
>            +-01.0
>            +-01.1
>            +-01.3
>            +-02.0
>            +-03.0
>            \-04.0
> # lspci -s 00:04.0 -xxxxx
> 00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
> 00: 86 80 ca 10 06 04 10 00 01 00 00 02 00 00 00 00
> 10: 04 00 01 f3 00 00 00 00 00 00 00 00 04 40 01 f3
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 3c a0
> 30: 00 00 00 00 70 00 00 00 00 00 00 00 00 00 00 00
> 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 70: 11 a0 02 80 03 00 00 00 03 20 00 00 00 00 00 00
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> a0: 10 00 02 00 c2 8c 00 00 10 28 00 00 41 6c 03 00
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> c0: 00 00 00 00 1f 00 00 00 00 00 00 00 00 00 00 00
> d0: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 
> -bash-4.1# more /vm-pci.cfg
> builder='hvm'
> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
> memory = 2048
> boot="d"
> maxvcpus=32
> vcpus=1
> serial='pty'
> vnclisten="0.0.0.0"
> name="latest"
> pci = ["0000:02:10.0"]
> 
> > 
> > Also to echo your second comment:  I might still be a newbie in qemu field 
> > (I started working on this 4 months ago).  I thought the chipset limits 
> > what you can see/do in vm.  Ie.  If you have 440fx emulations then you 
> > can't have any pci-e devices (fake or passthru) in the same system.  Is 
> > that not true?
> > 
> > Regards/Eniac
> > 
> > -----Original Message-----
> > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@xxxxxxxxxx]
> > Sent: Friday, February 21, 2014 5:32 PM
> > To: Zhang, Eniac
> > Cc: xen-devel@xxxxxxxxxxxxx
> > Subject: RE: [Xen-devel] q35 in xen? vfio in xen?
> > 
> > 
> > On Feb 21, 2014 4:58 PM, "Zhang, Eniac" <eniac-xw.zhang@xxxxxx> wrote:
> > >
> > > Hi Konrad,
> > >
> > > Thanks for your reply.
> > >
> > > Yes, I am aware of the pciback.  Unfortunately it doesn't seem to 
> > > support pci-e passthrough. (I could be wrong here)
> > 
> > I just did PCIe pass through of an VF of an SR-IOV device. It certainly is 
> > PCIe.
> > 
> > >
> > > There are two reason that I am interested in this.  For one, my project 
> > > calls for pci-e device passthrough, which can't be accomplished with 
> > > 440fx chipset emulation.  Secondly, I feel we ought to move on with the 
> > > technology.  440fx is ancient in computer terms.  Qemu is good and all 
> > > that, but if it refuses to support pci-e natively then it's just a matter 
> > > of time that it will become obsoleted.  The trend is clear that pci-e is 
> > > taking over the world. 
> > >
> > 
> > I am not sure what you are saying but it does not matter whether QEMU 
> > emulates 440fx or q35 for PCI pass through .
> > 
> > > Regards/Eniac
> > >
> > > -----Original Message-----
> > > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@xxxxxxxxxx]
> > > Sent: Friday, February 21, 2014 2:50 PM
> > > To: Zhang, Eniac
> > > Cc: xen-devel@xxxxxxxxxxxxx
> > > Subject: Re: [Xen-devel] q35 in xen? vfio in xen? 
> > >
> > > On Fri, Feb 21, 2014 at 09:41:39PM +0000, Zhang, Eniac wrote: 
> > > > Hi all,
> > > > 
> > > > I am playing with q35 chipset in qemu (1.6.1).  It seems we can't 
> > > > enable q35 machine under xen yet.  I made a few quick hacks which all 
> > > > fail miserably (linux kernel oops and window BSOD).  I was wondering 
> > > > why this hasn't been done (q35 was introduced into qemu in 2009). 
> > > > 
> > > > Next question, vfio works very well for me in standalone qemu (with 
> > > > Linux host handling iommu), but is that supported under xen?  I haven't 
> > > > tried anything there yet because my gut-feeling is that it won't work.  
> > > > Because passing vfio device to qemu can only be done on qemu 
> > > > commandline, and xen is not aware of this passing through device, thus 
> > > > not able to make iommu arrangement for this device.  Am I on the right 
> > > > track here? 
> > >
> > > Yes and no. VFIO won't work - but QEMU does do PCI passthrough under Xen. 
> > > It uses a different mechanism (and you need to bind the device to 
> > > pciback). 
> > >
> > > > 
> > > > I am interested in implementing both these two features.  I'd like to 
> > > > connect with anyone who's already on this so we don't duplicate the 
> > > > efforts. 
> > >
> > > What do you need Q35 for? 
> > >
> > > > 
> > > > Regards/Eniac
> > >
> > > > _______________________________________________
> > > > Xen-devel mailing list
> > > > Xen-devel@xxxxxxxxxxxxx
> > > > http://lists.xen.org/xen-devel
> > >

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel