Re: [Xen-users] Issues Booting DomU on TI DRA72 Chip

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

On Wed, 2015-07-15 at 08:37 -0400, Brandon Perez wrote:
> On 07/14/2015 04:43 AM, Ian Campbell wrote:
> > On Mon, 2015-07-13 at 08:33 -0400, Brandon Perez wrote:
> >> On 07/13/2015 04:29 AM, Ian Campbell wrote:
> >>> On Fri, 2015-07-10 at 11:21 -0400, Brandon Perez wrote:
> >>>
> >>> Is there a 1:1 mapping between (active) interrupts handled by this
> >>> cascaded controller a SPIs? i.e. if there were 32 interrupts but only 16
> >>> SPIs would you only be able to use 16 of the devices?
> >>>
> >>> Or does it support multiple interrupt sources triggering a single SPI?
> >>
> >>       For the chip I'm working with, it only supports a 1:1 mapping. It
> >> doesn't allow for multiple interrupts to share an SPI line.
> >
> > This simplifies things somewhat for passthrough then, I think. You
> > simply leave routing and assignment of interrupts to SPIs in the hands
> > of dom0, and on passthrough it would pass the SPI to the guest with
> > appropriate values for the interrupt-parent and interrupts properties.
> >
> >>> Right, this is a tricky one. Other than various solutions involving
> >>> trusting the firmware to get it right and requiring dom0 to not muck
> >>> around with it the only other solution I can think of would be a bit of
> >>> code in Xen which traps dom0's attempted changes and validates them,
> >>> preventing dom0 from mucking with the ones Xen cares about.
> >>>
> >>> Given how simple these sorts of interrupt remappers often are that might
> >>> not even be that much code.
> >>>
> >>> Ian.
> >>>
> >>
> >>       I think that wouldn't be too difficult to do, but for now, I'm
> >> going to stick with using the device tree to inform Xen and the Kernel.
> >
> > Sure.
> >
> 
> Hi Ian,
> 
>      This question pertains to what we've been talking about, so I'm 
> going to keep it on this thread.
> 
>      I was hoping to get some clarification on how secondary interrupts 
> are passed to Dom0 by Xen.

I think I should start by clarifying: Yours is the first platform we've
(or at least I've) come across which has this requirement and wants to
run Xen.

So the answer may we be something like "not well" or "my sheer
coincidence"...

>  My current understanding is as follows. When 
> Xen constructs Dom0, it goes through each entry in the device tree. It 
> then reads out each entry in the "interrupts" property. If the device is 
> connected to the primary interrupt controller, then Xen performs the 
> necessary steps to allocate the IRQ to the guest, and then route it to 
> the guest also. Otherwise, if the controller is not the primary one, 
> then Xen does nothing. This all happens in handle_device().

Yes, I think that's mostly right.

Recently I added support for handling the interrupt-map field for nodes
with device_type = "pci" (i.e. PCI buses), but this is limited to PCI.
It could perhaps be made more general.

>      Here's where my understanding becomes a little less clear. Later 
> on, the kernel will boot, and as needed, will write to its vGIC. This 
> will be trapped into Xen as a data abort, and handled as an MMIO write. 
> Eventually, it makes its way to the function gicv2_irq_enable(), which 
> actually writes to the vGIC and GIC to enable said interrupt. Does this 
> sound right, or am I missing some key components?

That sounds about correct, yes.

>      I'm attempting to enable the kernel to handle the mapping and 
> routing of IRQ's on the crossbar. So, what I'm doing is, for each device 
> in the DT that is a peripheral device (SPI), I'm changing its 
> "interrupt-parent" property  to be a device node representing the crossbar.

the "crossbar" is the thing which takes some pool of interrupts coming
from devices and maps them into specific SPIs in the GIC, is that right?

Does the node representing the cross-bar have the correct
interrupt-parent, i.e. pointing to the GIC?

If you change the interrupt-parent then I think you likely also need to
change the interrupts property, since it now needs to refer to things in
the "input irq space" of the cross bar and not the output (AKA SPIs).
The format for an interrupt-entry is defined by the interrupt-parent,
which includes the #interrupt-cells, but also the meaning of those
cells, which is defined in the relevant bindings.

e.g. for GIC it is
https://git.kernel.org/cgit/linux/kernel/git/devicetree/devicetree-rebasing.git/tree/Bindings/arm/gic.txt
 which declares that #interrupt-cells shall be 3 and they will be (type, nr, 
flags).

What I'm not sure about is how bindings are defined for interrupt
controllers which can map inputs to outputs dynamically, since the DT
cannot express that. Perhaps a specific driver is therefore needed.

If you are changing device tree things then you should take a look at
ePAPR which is the spec used on PPC which defined the framework for a
lot of this stuff. It might have some info about dynamic interrupts too.

>      However, this is leading to data aborts and/or interrupts not being 
> delivered to the kernel. Either the kernel encounters a data abort 
> (presumably due to the fact that it does not have permission for the 
> IRQ), or I find the kernel running in "idle_loop()" (which in my 
> experience means its waiting for an interrupt to be delivered).

I wouldn't expect a data abort from an access to the vgic, even if the
interrupt corresponding to the register isn't allowed to the guest.

Do you have your DT and kernel modifications working natively, i.e.
without Xen underneath?

>      Is the functionality in Xen to allow me to passthrough secondary 
> interrupts like this, or will some changes be needed?

Almost certainly changes will be needed.

> If so, could you point me in the right direction?

I can try, although it is getting outside the realms of my device-tree
knowledge it seems.

In any case it would be a lot easier to advise on the Xen specific
aspects if I could see the code changes and the device tree etc.

Ian.


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users