Re: [Xen-devel] [PATCH v5 20/19] docs: add "sched-gran" boot parameter documentation

[Jan Beulich <jbeulich@xxxxxxxx>]

On 30.09.2019 13:13, Jürgen Groß wrote:
> On 30.09.19 13:02, Jan Beulich wrote:
>> On 30.09.2019 12:51, Jürgen Groß wrote:
>>> On 30.09.19 12:25, Jan Beulich wrote:
>>>> On 30.09.2019 12:09, Juergen Gross wrote:
>>>>> Add documentation for the new "sched-gran" hypervisor boot parameter.
>>>>>
>>>>> Signed-off-by: Juergen Gross <jgross@xxxxxxxx>
>>>>> ---
>>>>>    docs/misc/xen-command-line.pandoc | 21 +++++++++++++++++++++
>>>>>    1 file changed, 21 insertions(+)
>>>>>
>>>>> diff --git a/docs/misc/xen-command-line.pandoc 
>>>>> b/docs/misc/xen-command-line.pandoc
>>>>> index fc64429064..c855246050 100644
>>>>> --- a/docs/misc/xen-command-line.pandoc
>>>>> +++ b/docs/misc/xen-command-line.pandoc
>>>>> @@ -1782,6 +1782,27 @@ Set the timeslice of the credit1 scheduler, in 
>>>>> milliseconds.  The
>>>>>    default is 30ms.  Reasonable values may include 10, 5, or even 1 for
>>>>>    very latency-sensitive workloads.
>>>>>    
>>>>> +### sched-gran (x86)
>>>>> +> `= cpu | core | socket`
>>>>> +
>>>>> +> Default: `sched-gran=cpu`
>>>>> +
>>>>> +Set the scheduling granularity. In case the granularity is larger than 1 
>>>>> (e.g.
>>>>> +`core`on a SMT-enabled system, or `socket`) multiple vcpus are assigned
>>>>> +statically to a "scheduling unit" which will then be subject to 
>>>>> scheduling.
>>>>> +This assignment of vcpus to scheduling units is fixed.
>>>>> +
>>>>> +`cpu`: Vcpus will be scheduled individually on single cpus.
>>>>> +
>>>>> +`core`: As many vcpus as there are hyperthreads on a physical core are
>>>>> +scheduled together on a physical core.
>>>>> +
>>>>> +`socket`: As many vcpus as there are hyperthreads on a physical sockets 
>>>>> are
>>>>> +scheduled together on a physical socket.
>>>>
>>>> I'd prefer if this didn't end up Intel-centric; ideally it also wouldn't be
>>>> x86-specific. AMD has introduced hyperthreading in Fam17 only; Fam15 used
>>>> "compute units", grouping together "cores". Internally the Intel side
>>>> "core vs hyperthread" is represented in the same variables 
>>>> (cpu_sibling_mask
>>>> in particular) as the AMD side "compute unit vs core".
>>>
>>> Yes, it is a mess.
>>>
>>>> Therefore it may be better to talk here about e.g. "smallest topological
>>>> sub-unit" and only say "e.g. a hyperthread to make a connection to common
>>>> x86 / Intel terminology". Of course the AMD side alternative use of the
>>>> variables also renders the actual command line option "sched-gran=core"
>>>> not overly fortunate. Perhaps we'd want to also use more abstract terms
>>>> here, e.g. topological "levels"?
>>>
>>> I think regarding usage of "hyperthreads" I'll go with:
>>>
>>> +`cpu`: Vcpus will be scheduled individually on single cpus (e.g. a
>>> + hyperthread using x86/Intel terminology)
>>> +
>>> + `core`: As many vcpus as there are cpus on a physical core are
>>> + scheduled together on a physical core.
>>> ...
>>>
>>> I think using "core" is fine. We have it in multiple places in the
>>> hypervisor which are _not_ specific to Intel.
>>
>> Well, what we have in hypervisor sources is one thing - we can
>> settle on any convention we want there. It's the user (admin)
>> interface (i.e. the command line option name and description
>> here) which we may want to be a little more careful with. But
>> yes, I can see how we use "core" already in similar contexts
>> in the command line option doc, first and foremost on
>> "credit2_runqueue". (In retrospect I think this might have been
>> a mistake though.)
> 
> So what do you suggest?
> 
> <Irony on>
> "topology-level-just-above-the-smallest-topological-sub-unit"?
> <Irony-off>
> 
> I can't think of any sensible terminology not resulting in something
> which is much harder to understand than "core".

Ideally I'd like us to have an arch-independent way of
expressing things - "socket" and "node" look to be common enough,
so perhaps wouldn't need further abstraction, but sub-socket
granularities could perhaps be expressed as "level1" or "level2"?
And then there could be context sensitive meanings of "core",
"cu", and perhaps (in the future) "die".

My concern is that AMD-focused people may, when using "core", not
get what they'd expect (and this concern extends to the existing
uses of "core"). IOW "context sensitive" above would assign
different meaning to "core" depending on the hardware we run on.
Granted I can also see how this might confuse people other than
the example AMD-focused ones.

> And we are using "core" or "cores" in hypervisor messages, too.

That's still slightly different though.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel