Re: [Xen-devel] [Xen-users] Grant iomem access and map IRQs to a domU guest in Xen for ARM targets

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

(adding xen-devel for some of these issues)

On Tue, 2014-06-10 at 01:37 +0000, Kapania, Ashish wrote:
> > -----Original Message-----
> > From: Ian Campbell [mailto:Ian.Campbell@xxxxxxxxxx]
> > Sent: Monday, June 09, 2014 1:42 AM
> > To: Kapania, Ashish
> > Cc: xen-users@xxxxxxxxxxxxx
> > Subject: Re: [Xen-users] Grant iomem access and map IRQs to a domU
> > guest in Xen for ARM targets
> > 
> > On Fri, 2014-06-06 at 20:25 +0000, Kapania, Ashish wrote:
> > > Hi All,
> > >
> > > I am working on running a RTOS as a domU guest on a omap5 evm
> > (Cortex-A15).
> > > As part of this effort, I need to create drivers for certain
> > > peripherals that are owned by the domU guest (basically make the domU
> > > a driver domain for certain peripherals). In order to achieve this I
> > > need to be able to grant domU access to certain memory mapped
> > > registers and map a SPI IRQ generated by the hw peripheral to it.
> > >
> > > I went through the xl.cfg documentation and my understanding is that
> > > it supports 2 options called "iomem" and "irqs" that I can use to
> > > achieve the aforementioned purpose when creating a guest. However, it
> > > looks like these are not implemented in Xen-ARM. My question is that
> > > what is the recommended way of achieving this in Xen for ARM targets
> > ?
> > 
> > Support for iomem= is being worked on right now. See patches from
> > Arianna Avanzini on xen-devel over the last few months. Julien Grall is
> > also working on irq passthrough which I think will integrate irqs=
> > support but also a "higher level" ability to passthrough a device based
> > on e.g. device tree nodes without having to worry about the specific
> > resources. I'm not sure when this will be ready though.
> > 
> > In the meantime (and with the Xen 4.4 release) people who want this
> > functionality have just been hacking the hypervisor to add the specific
> > mappings which they need to the domU domains. The key functions in the
> > hypervisor are map_mmio_regions and route_irq_to_guest (see
> > xen/arch/arm/platforms/*.c for examples of using these to route devices
> > which are not described in devicetree to dom0, which is a bit similar
> > to what you want)
> > 
> > I'm not sure how people have been triggering these extra mappings. I
> > suppose you could either do it statically for each domid==N e.g. in
> > arch_domain_create (but then you can't easily restart the guests) or
> > you could add a custom domctl and add a call in the toolstack, or a new
> > DOMCRF_ flag (which gets passed to arch_domain_create).
> > 
> 
> Thanks Ian, this helps. I think I am going to go ahead with your
> Suggestion to create a custom domctl op for now and add a call to it in
> the xen toolstack. This should allow me to specify the desired mappings
> in the domU.cfg file.

Great.

> > Are the devices you wish to pass through DMA capable? Unless they are
> > being an SMMU that will make things more complex.
> > 
> 
> At the moment the peripherals I need to pass through are not DMA capable.
> In the future however, I may have to pass through a dma-capable device.
> It will all depend on the use-cases that we will need to support on our
> RTOS.
> 
> I believe the TI chip I am working on (OMAP5) does not have a SMMU.

I thought it did have some sort of IOMMU (perhaps not exactly an SMMU)
for some peripherals, but I've not worked with or even investigated the
platform much.

> Considering the lack of SMMU, if I need to pass through a dma-capable
> device, how do you recommend I go about it ? Do I need to implement
> something similar to linux's swiotlb-xen driver in our RTOS ?

It depends on the nature of the specific device.

One approach (the preferred one I think) would be to create a PV
front/back pair for the class of devices to allow dom0 to drive the
hardware and provide mediated access to it to guests (assuming such a
driver pair doesn't already exist). There is talk [0,1] of perhaps
starting a new subproject with the aim of providing PV drivers for
various "embedded" type devices (and I think OMAP is one of the
platforms used by the people involved).

Another option would be some sort of software SMMU or swiotlb thing like
you've suggested. I think that would involve a small amount of
hypervisor support to map guest IPAs in to DMA address (i.e. real PAs).
There have been proposals along those lines e.g. [2] which I think deals
with IPU stuff on OMAP.

Lastly the 1:1 workaround could potentially be enabled for other guests,
but that is a pretty nasty hack and would cause potential issues e.g.
with rebooting such a guest (if it couldn't allocate a contiguous region
next time).

Oh, and I suppose it might be possible to not use the DMA capabilities
of the h/w, but I guess that would be pretty awful for perf and power
etc, you would probably not want to go that path I suppose.

Ian.

[0] http://lists.xen.org/archives/html/xen-devel/2014-06/msg00444.html
[1] http://lists.xen.org/archives/html/xen-devel/2014-06/msg00289.html
[2] http://lists.xen.org/archives/html/xen-devel/2014-01/msg01913.html


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel