Re: [Xen-devel] [Query] Flow of PCI device dedicated to a domU

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

Please send email in plain text.


On Thu, 24 Jul 2014, Xenbegn developer wrote:
> On Tue, Jul 22, 2014 at 3:20 PM, Xenbegn developer <xen.begn.dev@xxxxxxxxx> 
> wrote:
> 
> 
> 
>       On Wed, Jul 16, 2014 at 3:20 PM, Andrew Cooper 
> <andrew.cooper3@xxxxxxxxxx> wrote:
>             On 16/07/14 07:12, Xenbegn developer wrote:
>       Hi,
> I am trying to understand the flow of how a PCI device driver in domU works 
> after a PCI device is
> is assigned to a domU.
> 
> a) If a PCI device is assigned to a domU, this device has to be on a PCI bus. 
> So as per my view xen
> would have to somehow provide a PCI Controller on which this device is 
> attached.
> => Is my assumption correct ? If yes how it is done, No then also How 
> enumeration of this device
> happens in domU kernel
> 
> 
> No.
> 
> PV guests have no PCI root ports/bridges; they use devices as single entities 
> knowing that Xen/dom0 takes
> care of the other bits. HVM guests have their PCI devices attached to the 
> virtual southbridge which is
> all emulated by Qemu.
> 
> 
> b) Is the Configuration space of the PCI device directly accessible to the 
> domU (assuming the
> kernel accesses it using memory map) ? If not then it is trapped by xen
> 
> 
> All configuration space is trap+emulate in Xen, although almost all 
> operations permitted.
> 
> 
> c) Who assigns the MSI (addr + value) in the PCI device. If Xen then how Xen 
> does a translation
> from Physical MSI to guest MSI (where in code)
> 
> 
> Xen controls all interrupts on the system, which is why it needs to trap all 
> config accesses to notice
> when a domain is attempting to change the interrupt information. In that 
> case, Xen fixes up its delivery
> of interrupts to the guest, but leaves the underlying interrupt information 
> intact.
> 
> ~Andrew
> 
> 
> Linux kernel pci code has a function called __write_msi_msg. This method 
> writes to config space for MSI/MSIX
> pci_write_config_dword(dev, pos + PCI_MSI_ADDRESS_LO,
> ÂÂÂ ÂÂÂ ÂÂÂ ÂÂÂ ÂÂÂÂÂÂ msg->address_lo);
> 
> Now as per (b) and (c) it has to be trapped in xen. But in x86/traps.c 
> (guest_io_write) I dont see any MSI
> handling.
> Can you please explain (point to the code) where domain is attempting to 
> change the interrupt information.
> ÂÂÂ
> 
> Stefano, Can you please comment on it.

Keep in mind that on x86 there are two ways to do PCI passthrough:

1) HVM guests, the PCI bridge is emulated, PCI accesses are trapped

2) PV guests, pcifront/pciback drivers are used, no emulation is done,
   no trapping

In ARM we would do 2). In the (2) case, x86/traps.c is not involved.

The guest enables MSIs via pcifront. pcifront would send the request to
pciback that would actually enable MSIs on the physical device.  See
drivers/xen/xen-pciback/pciback_ops.c:xen_pcibk_enable_msi.  The guest
kernel would map the MSI into a "pirq" to receive MSI notifications as
event channels.  No trapping is needed. See:
xen/arch/x86/msi.c:xen_setup_msi_irqs.  Event channels are a Xen
software interrupt mechanism, see drivers/xen/evtchn.c and
drivers/xen/events.  On ARM we would probably deliver physical MSIs as
virtual MSIs, thanks to the capabilities of the vgic.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel