Re: Keystone Issue

[Bertrand Marquis <Bertrand.Marquis@xxxxxxx>]

Hi,

> On 9 Jun 2020, at 21:07, CodeWiz2280 <codewiz2280@xxxxxxxxx> wrote:
> 
> On Tue, Jun 9, 2020 at 1:45 PM Marc Zyngier <maz@xxxxxxxxxx> wrote:
>> 
>> Hi Julien,
>> 
>> On 2020-06-09 18:32, Julien Grall wrote:
>>> (+ Marc)
>>> 
>>> On 09/06/2020 18:03, Bertrand Marquis wrote:
>>>> Hi
>>>> 
>>>>> On 9 Jun 2020, at 16:47, Julien Grall <julien@xxxxxxx> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>> On 09/06/2020 16:28, Bertrand Marquis wrote:
>>>>>> Hi,
>>>>>>> On 9 Jun 2020, at 15:33, CodeWiz2280 <codewiz2280@xxxxxxxxx> wrote:
>>>>>>> 
>>>>>>> There does appear to be a secondary (CIC) controller that can
>>>>>>> forward
>>>>>>> events to the GIC-400 and EDMA controllers for the keystone 2
>>>>>>> family.
>>>>>>> Admittedly, i'm not sure how it is being used with regards to the
>>>>>>> peripherals.  I only see mention of the GIC-400 parent for the
>>>>>>> devices
>>>>>>> in the device tree.  Maybe Bertrand has a better idea on whether
>>>>>>> any
>>>>>>> peripherals go through the CIC first?  I see that gic_interrupt ()
>>>>>>> fires once in Xen, which calls doIRQ to push out the virtual
>>>>>>> interrupt
>>>>>>> to the dom0 kernel.  The dom0 kernel then handles the interrupt and
>>>>>>> returns, but gic_interrupt() never fires again in Xen.
>>>>>> I do not remember of any CIC but the behaviour definitely look like
>>>>>> an interrupt acknowledge problem.
>>>>>> Could you try the following:
>>>>>> --- a/xen/arch/arm/gic-v2.c
>>>>>> +++ b/xen/arch/arm/gic-v2.c
>>>>>> @@ -667,6 +667,9 @@ static void gicv2_guest_irq_end(struct irq_desc
>>>>>> *desc)
>>>>>>      /* Lower the priority of the IRQ */
>>>>>>      gicv2_eoi_irq(desc);
>>>>>>      /* Deactivation happens in maintenance interrupt / via GICV */
>>>>>> +
>>>>>> +    /* Test for Keystone2 */
>>>>>> +    gicv2_dir_irq(desc);
>>>>>>  }
>>>>>> I think the problem I had was related to the vgic not deactivating
>>>>>> properly the interrupt.
>>>>> 
>>>>> Are you suggesting the guest EOI is not properly forwarded to the
>>>>> hardware when LR.HW is set? If so, this could possibly be workaround
>>>>> in Xen by raising a maintenance interrupt every time a guest EOI an
>>>>> interrupt.
>>>> 
>>>> Agree the maintenance interrupt would definitely be the right solution
>>> I would like to make sure we aren't missing anything in Xen first.
>>> From what you said, you have encountered this issue in the past with a
>>> different hypervisor. So it doesn't look like to be Xen related.
>>> 
>>> Was there any official statement from TI? If not, can we try to get
>>> some input from them first?
> Thank you all for your support so far, its really appreciated.  Is
> there a quick patch that I can try with this maintenance interrupt to
> get the level interrupts working as well? I can pose the question to
> TI but would like to close the loop and make sure there are no other
> issues that pop out first.

If you can that would be good to ask TI because I did work on the Keystone2 a 
while ago and they might have a firmware solution for that.

Bertrand

>>> 
>>> @Marc, I know you dropped 32-bit support in KVM recently :). Although,
>> 
>> Yes! Victory is mine! Freedom from the shackles of 32bit, at last! :D
>> 
>>> I was wondering if you heard about any potential issue with guest EOI
>>> not forwarded to the host. This is on TI Keystone (Cortex A-15).
>> 
>> Not that I know of. A-15 definitely works (TC2, Tegra-K1, Calxeda Midway
>> all run just fine with guest EOI), and GIC-400 is a pretty solid piece
>> of kit (it is just sloooooow...).
>> 
>> Thinking of it, you would see something like that if the GIC was seeing
>> the writes coming from the guest as secure instead of NS (cue the early
>> firmware on XGene that exposed the wrong side of GIC-400).
>> 
>> Is there some kind of funky bridge between the CPU and the GIC?
>> 
>>         M.
>> --
>> Jazz is not dead. It just smells funny...