Re: [Xen-devel] [RFC 0/5] xen/arm: support big.little SoC

[Peng Fan <van.freenix@xxxxxxxxx>]

On Wed, Sep 21, 2016 at 10:22:14AM +0100, George Dunlap wrote:
>On 21/09/16 09:38, Peng Fan wrote:
>> On Tue, Sep 20, 2016 at 01:17:04PM -0700, Stefano Stabellini wrote:
>>> On Tue, 20 Sep 2016, Julien Grall wrote:
>>>> Hi Stefano,
>>>>
>>>> On 20/09/2016 20:09, Stefano Stabellini wrote:
>>>>> On Tue, 20 Sep 2016, Julien Grall wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On 20/09/2016 12:27, George Dunlap wrote:
>>>>>>> On Tue, Sep 20, 2016 at 11:03 AM, Peng Fan <van.freenix@xxxxxxxxx>
>>>>>>> wrote:
>>>>>>>> On Tue, Sep 20, 2016 at 02:54:06AM +0200, Dario Faggioli wrote:
>>>>>>>>> On Mon, 2016-09-19 at 17:01 -0700, Stefano Stabellini wrote:
>>>>>>>>>> On Tue, 20 Sep 2016, Dario Faggioli wrote:
>>>>>>>> I'd like to add a computing capability in xen/arm, like this:
>>>>>>>>
>>>>>>>> struct compute_capatiliby
>>>>>>>> {
>>>>>>>>    char *core_name;
>>>>>>>>    uint32_t rank;
>>>>>>>>    uint32_t cpu_partnum;
>>>>>>>> };
>>>>>>>>
>>>>>>>> struct compute_capatiliby cc=
>>>>>>>> {
>>>>>>>>   {"A72", 4, 0xd08},
>>>>>>>>   {"A57", 3, 0xxxx},
>>>>>>>>   {"A53", 2, 0xd03},
>>>>>>>>   {"A35", 1, ...},
>>>>>>>> }
>>>>>>>>
>>>>>>>> Then when identify cpu, we decide which cpu is big and which cpu is
>>>>>>>> little
>>>>>>>> according to the computing rank.
>>>>>>>>
>>>>>>>> Any comments?
>>>>>>>
>>>>>>> I think we definitely need to have Xen have some kind of idea the
>>>>>>> order between processors, so that the user doesn't need to figure out
>>>>>>> which class / pool is big and which pool is LITTLE.  Whether this sort
>>>>>>> of enumeration is the best way to do that I'll let Julien and Stefano
>>>>>>> give their opinion.
>>>>>>
>>>>>> I don't think an hardcoded list of processor in Xen is the right 
>>>>>> solution.
>>>>>> There are many existing processors and combinations for big.LITTLE so it
>>>>>> will
>>>>>> nearly be impossible to keep updated.
>>>>>>
>>>>>> I would expect the firmware table (device tree, ACPI) to provide relevant
>>>>>> data
>>>>>> for each processor and differentiate big from LITTLE core.
>>>>>> Note that I haven't looked at it for now. A good place to start is 
>>>>>> looking
>>>>>> at
>>>>>> how Linux does.
>>>>>
>>>>> That's right, see Documentation/devicetree/bindings/arm/cpus.txt. It is
>>>>> trivial to identify the two different CPU classes and which cores belong
>>>>> to which class.t, as
>>>>
>>>> The class of the CPU can be found from the MIDR, there is no need to use 
>>>> the
>>>> device tree/acpi for that. Note that I don't think there is an easy way in
>>>> ACPI (i.e not in AML) to find out the class.
>>>>
>>>>> It is harder to figure out which one is supposed to be
>>>>> big and which one LITTLE. Regardless, we could default to using the
>>>>> first cluster (usually big), which is also the cluster of the boot cpu,
>>>>> and utilize the second cluster only when the user demands it.
>>>>
>>>> Why do you think the boot CPU will usually be a big one? In the case of 
>>>> Juno
>>>> platform it is configurable, and the boot CPU is a little core on r2 by
>>>> default.
>>>>
>>>> In any case, what we care about is differentiate between two set of CPUs. I
>>>> don't think Xen should care about migrating a guest vCPU between big and
>>>> LITTLE cpus. So I am not sure why we would want to know that.
>>>
>>> No, it is not about migrating (at least yet). It is about giving useful
>>> information to the user. It would be nice if the user had to choose
>>> between "big" and "LITTLE" rather than "class 0x1" and "class 0x100", or
>>> even "A7" or "A15".
>> 
>> As Dario mentioned in previous email,
>> for dom0 provide like this:
>> 
>> dom0_vcpus_big = 4
>> dom0_vcpus_little = 2
>> 
>> to dom0.
>> 
>> If these two no provided, we could let dom0 runs on big pcpus or big.little.
>> Anyway this is not the important point for dom0 only big or big.little.
>> 
>> For domU, provide "vcpus.big" and "vcpus.little" in xl configuration file.
>> Such as:
>> 
>> vcpus.big = 2
>> vcpus.litle = 4
>
>FWIW, from a UI perspective, it would be nice if we designed the
>interface such that it *can* be used simply (i.e., just "big" or
>"little"), but can also be used more flexibly; for instance, specifying
>"A15" or "A7" instead.
>
>So maybe have a 'classifier' string; this could start by having just
>"big" and "little", but could then be extended to allow fuller ways of
>specifying specific kinds of cores.
>
>To keep the illusion of python syntax, what about something like this:
>
>vcpuclass=["big=2","little=4"]
>
>Or would it be better to have a mapping of vcpu to class?
>
>vcpuclass=["0-1:big","2-5:little"]


Both are good -:)

>
>
>> According to George's comments,
>> Then, I think we could use affinity to restrict little vcpus be scheduled on 
>> little vcpus,
>> and restrict big vcpus on big vcpus. Seems no need to consider soft 
>> affinity, use hard
>> affinity is to handle this.
>> 
>> We may need to provide some interface to let xl can get the information such 
>> as
>> big.little or smp. if it is big.little, which is big and which is little.
>
>If it's possible for Xen to order the cpus by class, then there could be
>a hypercall listing the different classes starting with the largest
>class.  On typical big.LITTLE systems, class 0 would be "big" and class
>1 would be "little".

From Class 0 to Class X, the computing capacity or dmips is decreasing -:)

Regards,
Peng.
>
>
>> User may change the hard affinity of a vcpu, so we also need to block a 
>> little
>> vcpu be scheduled to a big physical cpu. Add some checking code in xen,
>> when chaning the hard affnity, check whether the cap of a vcpu is compatible
>> with the cap of the physical cpus.
>
>Dario, what do we do with vNUMA / soft affinity?
>
> -George

-- 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel