Re: [Xen-devel] [Xen-users] Grant iomem access and map IRQs to a domU guest in Xen for ARM targets

["Kapania, Ashish" <akapania@xxxxxx>]

> -----Original Message-----
> From: Ian Campbell [mailto:Ian.Campbell@xxxxxxxxxx]
> Sent: Tuesday, June 10, 2014 2:24 AM
> To: Kapania, Ashish
> Cc: xen-users@xxxxxxxxxxxxx; xen-devel
> Subject: Re: [Xen-users] Grant iomem access and map IRQs to a domU
> guest in Xen for ARM targets
> 
> (adding xen-devel for some of these issues)
> 
> On Tue, 2014-06-10 at 01:37 +0000, Kapania, Ashish wrote:
> > > -----Original Message-----
> > > From: Ian Campbell [mailto:Ian.Campbell@xxxxxxxxxx]
> > > Sent: Monday, June 09, 2014 1:42 AM
> > > To: Kapania, Ashish
> > > Cc: xen-users@xxxxxxxxxxxxx
> > > Subject: Re: [Xen-users] Grant iomem access and map IRQs to a domU
> > > guest in Xen for ARM targets
> > >
> > > On Fri, 2014-06-06 at 20:25 +0000, Kapania, Ashish wrote:
> > > > Hi All,
> > > >
> > > > I am working on running a RTOS as a domU guest on a omap5 evm
> > > (Cortex-A15).
> > > > As part of this effort, I need to create drivers for certain
> > > > peripherals that are owned by the domU guest (basically make the
> > > > domU a driver domain for certain peripherals). In order to
> achieve
> > > > this I need to be able to grant domU access to certain memory
> > > > mapped registers and map a SPI IRQ generated by the hw peripheral
> to it.
> > > >
> > > > I went through the xl.cfg documentation and my understanding is
> > > > that it supports 2 options called "iomem" and "irqs" that I can
> > > > use to achieve the aforementioned purpose when creating a guest.
> > > > However, it looks like these are not implemented in Xen-ARM. My
> > > > question is that what is the recommended way of achieving this in
> > > > Xen for ARM targets
> > > ?
> > >
> > > Support for iomem= is being worked on right now. See patches from
> > > Arianna Avanzini on xen-devel over the last few months. Julien
> Grall
> > > is also working on irq passthrough which I think will integrate
> > > irqs= support but also a "higher level" ability to passthrough a
> > > device based on e.g. device tree nodes without having to worry
> about
> > > the specific resources. I'm not sure when this will be ready
> though.
> > >
> > > In the meantime (and with the Xen 4.4 release) people who want this
> > > functionality have just been hacking the hypervisor to add the
> > > specific mappings which they need to the domU domains. The key
> > > functions in the hypervisor are map_mmio_regions and
> > > route_irq_to_guest (see xen/arch/arm/platforms/*.c for examples of
> > > using these to route devices which are not described in devicetree
> > > to dom0, which is a bit similar to what you want)
> > >
> > > I'm not sure how people have been triggering these extra mappings.
> I
> > > suppose you could either do it statically for each domid==N e.g. in
> > > arch_domain_create (but then you can't easily restart the guests)
> or
> > > you could add a custom domctl and add a call in the toolstack, or a
> > > new DOMCRF_ flag (which gets passed to arch_domain_create).
> > >
> >
> > Thanks Ian, this helps. I think I am going to go ahead with your
> > Suggestion to create a custom domctl op for now and add a call to it
> > in the xen toolstack. This should allow me to specify the desired
> > mappings in the domU.cfg file.
> 
> Great.
> 
> > > Are the devices you wish to pass through DMA capable? Unless they
> > > are being an SMMU that will make things more complex.
> > >
> >
> > At the moment the peripherals I need to pass through are not DMA
> capable.
> > In the future however, I may have to pass through a dma-capable
> device.
> > It will all depend on the use-cases that we will need to support on
> > our RTOS.
> >
> > I believe the TI chip I am working on (OMAP5) does not have a SMMU.
> 
> I thought it did have some sort of IOMMU (perhaps not exactly an SMMU)
> for some peripherals, but I've not worked with or even investigated the
> platform much.
>
> > Considering the lack of SMMU, if I need to pass through a dma-capable
> > device, how do you recommend I go about it ? Do I need to implement
> > something similar to linux's swiotlb-xen driver in our RTOS ?
> 
> It depends on the nature of the specific device.
>
> One approach (the preferred one I think) would be to create a PV
> front/back pair for the class of devices to allow dom0 to drive the
> hardware and provide mediated access to it to guests (assuming such a
> driver pair doesn't already exist). There is talk [0,1] of perhaps
> starting a new subproject with the aim of providing PV drivers for
> various "embedded" type devices (and I think OMAP is one of the
> platforms used by the people involved).
> 
> Another option would be some sort of software SMMU or swiotlb thing
> like you've suggested. I think that would involve a small amount of
> hypervisor support to map guest IPAs in to DMA address (i.e. real PAs).
> There have been proposals along those lines e.g. [2] which I think
> deals with IPU stuff on OMAP.
>

[2] seems useful if we need to get shared memory IPC working between
domU guest and the remote processors on the SoC.

I will try to stick to using PV front/back end pair to provide domU
access to peripherals unless performance requirements necessitate
running a driver directly on domU. In which case it seems my best
bet is emulating SMMU in software.

Thanks for your inputs.

Best,
Ashish
 
> Lastly the 1:1 workaround could potentially be enabled for other
> guests, but that is a pretty nasty hack and would cause potential
> issues e.g.
> with rebooting such a guest (if it couldn't allocate a contiguous
> region next time).
> 
> Oh, and I suppose it might be possible to not use the DMA capabilities
> of the h/w, but I guess that would be pretty awful for perf and power
> etc, you would probably not want to go that path I suppose.
> 
> Ian.
> 
> [0] http://lists.xen.org/archives/html/xen-devel/2014-06/msg00444.html
> [1] http://lists.xen.org/archives/html/xen-devel/2014-06/msg00289.html
> [2] http://lists.xen.org/archives/html/xen-devel/2014-01/msg01913.html

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel