Re: [Xen-devel] [PATCH v1 4/4] xen/arm: Reuse R-Car Gen2 platform code for Stout board

[Oleksandr Tyshchenko <olekstysh@xxxxxxxxx>]

On Fri, Aug 10, 2018 at 12:44 PM, Julien Grall <julien.grall@xxxxxxx> wrote:
> On 08/09/2018 07:18 PM, Oleksandr Tyshchenko wrote:
>>
>> Hi Julien.
>
>
> Hi Oleksandr,
Hi Julien

>
>> On Thu, Aug 9, 2018 at 7:20 PM, Julien Grall <julien.grall@xxxxxxx> wrote:
>>>
>>>
>>>
>>> On 08/09/2018 05:18 PM, Oleksandr Tyshchenko wrote:
>>>>
>>>>
>>>> On Thu, Aug 9, 2018 at 7:10 PM, Julien Grall <julien.grall@xxxxxxx>
>>>> wrote:
>>>>>
>>>>>
>>>>> Somehow I thought the platform was 64-bit and found a SOC name very
>>>>> similar
>>>>> to it. Sorry for the confusion. PSCI seems indeed not supported for
>>>>> that
>>>>> platform.
>>>>
>>>>
>>>> R-Car Gen3 is ARM64 (H2 SoC -> r8a7790) and does support PSCI.
>>>> But R-Car Gen2 is ARM32 (H2 SoC -> r8a7790)
>>>>
>>>>>
>>>>> However, the code looks fairly different than what we have in Xen. For
>>>>> instance secondary CPU seems to require to initialize CNTVOFF, the
>>>>> function
>>>>> to power on a CPU also looks different.
>>>>
>>>>
>>>> Sorry, which code you are taking about, U-Boot or Linux?
>>>
>>>
>>>
>>> The SMP code in Linux vs the SMP code in Xen for the R-Car. In most of
>>> the
>>> cases, Xen should replicate what Linux does.
>>>
>>> I am trying to understand why such differences at the moment.
>>
>>
>> As I understand, the SMP code for Linux can't be used in Xen, since
>> the requirement is to boot into Xen with cores already being in
>> Hypervisor mode.
>> But, all cores seems to start in Monitor mode after powering them up.
>> So, they must be switched into Hypervisor mode beforehand, correct?
>
>
> I am quite confused here. If a CPU will be in hypervisor mode for Xen, how
> come the same CPU will boot in monitor mode for Linux? Surely there should
> be no difference in term of boot here. Are you using different firmware
> here?

Yes, in order to run modified U-Boot is used. You can see here what
exactly was modified.
This instruction [1] contains link to patches [2] (see "Apply
additional patches" step)
which should be applied to bare U-Boot.

[1] 
https://wiki.xenproject.org/wiki/Xen_ARM_with_Virtualization_Extensions/Lager
[2] https://lists.xenproject.org/archives/html/xen-devel/2015-01/msg02475.html

>
>>
>> U-Boot brings non-boot cores up (using ported from Linux's
>> platsmp-apmu code functions). Then U-Boot switches all cores to HYP
>> mode.
>
>
> Who did the port? Is it upstream?

AFAIR Iurii Konovalenko did this port, he is also an author of
xen/arch/arm/platforms/rcar2.c.
I don't think U-Boot changes are upsteamed. But anybody who wants to
run Xen on Lager board is able (I think) to apply
published patches [2] to U-Boot and build it.
The same way anybody who will want to run Xen on Stout board will be able
to apply U-Boot patches I will attach to the future Wiki page for Stout board.

>
>> While jump into Xen on a boot core, all non-boot cores are left in
>> U-Boot to wait to be "woken up by Xen" (actually rcar2_smp_init() is
>> used for).
>> Platform R-Car2 code in Xen doesn't power on cores, it just "brings
>> them to Xen".
>
>
> What does the address 0xe63c0000 corresponds too? From the DT it seems it is
> part of the ICRAM. But it is not clear why you write in there.
Yes, it is a part of the ICRAM.

Unfortunately, I am not an expert in such code to definitively say how
it is supposed to work.
But it seems that 0xE63C0FFC is an address which is being polled by
non-boot core in U-Boot. It was set to zero value
beforehand. So, writing an actual value to this address, Xen triggers
an action for bringing non-boot core up to Xen.

Waiting loop in U-Boot:

asm(".arm \n"
".align 2 \n"
".global smp_waitloop \n"
"smp_waitloop: \n"
"1: wfe \n"
"ldr r0, =0xE63C0FFC \n"
"ldr r0, [r0] \n"
"teq r0, #0x0 \n"
"beq 1b \n"
  "b _do_nonsec_entry \n"
".type smp_waitloop, %function \n"
".size smp_waitloop, .-smp_waitloop \n");

Trigger code in Xen:

writel(__pa(init_secondary), 0xE63C0FFC);

>
> Could you point to any relevant documentation?
I am afraid, I can't point to documentation.

>
> Cheers,
>
> --
> Julien Grall



-- 
Regards,

Oleksandr Tyshchenko

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel