Re: [Xen-devel] SR-IOV problems - HVM cannot access network

[David White <dwhite@xxxxxxxxxxxxx>]

On 03/01/2011 02:41 PM, Rose, Gregory V wrote:
> > -----Original Message-----
> > From: David White [mailto:dwhite@xxxxxxxxxxxxx]
> > Sent: Tuesday, March 01, 2011 2:05 PM
> > To: Rose, Gregory V
> > Cc:xen-devel@xxxxxxxxxxxxxxxxxxx
> > Subject: Re: [Xen-devel] SR-IOV problems - HVM cannot access network
>   [snip}
>
> > The guest driver does not seem to be the source of this difference,
> > since these qemu logs are logged before HVM grub menu appears (and hence
> > before guest kernel is loaded)
> >
> > Why would a physical function invoke MSI-X but not a virtual function?
> A lot of reasons actually :-).  SR-IOV is a complete platform solution and
> feature.  It requires full support from all PCIe devices from the root complex
> to the endpoint device as well as proper BIOS programming and OS support.
>
> I don't know that Ubuntu actually supports SR-IOV and I don't know which
> platform you're using and whether it supports SR-IOV.
>
> > Is it something that pciback is doing?
> >
> > When I bind a PF to pciback, the xen-pciback driver shows:
> > [ 1390.090693] pciback 0000:04:00.1: seizing device
> > [ 1390.095417] xen_allocate_pirq: returning irq 17 for gsi 17
> > [ 1390.101021] Already setup the GSI :17
> > [ 1390.104717] pciback 0000:04:00.1: PCI INT B ->  GSI 17 (level, low) ->
> > IRQ 17
> > [ 1390.111807] pciback 0000:04:00.1: PCI INT B disabled
> >
> > but when I bind a VF to pciback, the driver shows:
> > [ 1439.411763] pciback 0000:04:10.0: seizing device
> > [ 1439.416462] pciback 0000:04:10.0: enabling device (0000 ->  0002)
> >
> > (note no INT or GSI messages)
> >
> > could pciback be the source of the HVM INTx problem?
> Well it's certainly possible but another good possibility is that the machine
> doesn't actually implement the SR-IOV feature correctly in the BIOS.  What
> machine are you using?  Is there a BIOS setting to enable SR-IOV and if so are
> you sure you've enabled it?
>
> Also, would it be possible to get a register dump of the VF device from both
> the Dom0 host domain and the DomU guest domain using the Intel ethregs
> utility?  You can get that from source forge.
>
> - Greg
Thanks for your help so far.

Ubuntu Maverick is only running in the guest HVM domains.

I have two boards I am testing with, both with Intel 5520 chipset.  I'm using
two 82576 nics from two different vendors, one attached to either board.  Both
systems have AMI bios.  One system (call it systemA) has bios options for both
VT-d and SR-IOV, and both are enabled, as is I/OAT.  The other system (systemB)
has bios options for VT-d and "Coherency support" (which seemed to help me
enumerate the VFs), and both are enabled (no mention of I/OAT).

SystemA runs Debian Squeeze and the xen kernel, vmm and tools from that distro.
SystemB runs Fedora14 with the well-known "jeremy" kernel
(http://git.kernel.org/?p=linux/kernel/git/jeremy/xen.git;a=shortlog;h=xen/stable-2.6.32.x),
and xen-4.0.1 hypervisor built from source (unaltered).

Both systems are experiencing the same behavior.

attached are dom0 and hvm ethregs output for the VF using '-o -m'
flags.  This one was taken from the Debian host ("systemA")

-dave

Attachment: ethregs.VF.dom0
Description: Text document

Attachment: ethregs.VF.hvm
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel