Re: [Xen-devel] multi-core VMM

[François-Frédéric Ozog <ff@xxxxxxxx>]

Before SandyBridge, it was not a good idea to have two mem bounded threads
run on a single core: there was only ONE memory read port. On SandyBridge,
IvvyBridge and Haswell, there are more memory ports and L3 cache is split
between cores. Each core has a "CBo" (cache box) indirectly attached to
2.5MB of L3 cache. A consistent hash function across cores is calculated on
cacheline address to obtain the target CBo. So there are no "locality"
benefit from this chunking.

I can't see a way for schedulers to decide the policy, that would be an
application developer controlled thing.
So I would rather leverage HT information to actually make sure some threads
are NOT on the same core rather than trying to group them.
This also means that VCPU to CPU mapping needs to be controlled tightly...

François-Frédéric

-----Message d'origine-----
De : xen-devel-bounces@xxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxx] De la part de G.R.
Envoyé : jeudi 10 janvier 2013 16:25
À : ZHANG Zhi
Cc : xen-devel@xxxxxxxxxxxxx
Objet : Re: [Xen-devel] multi-core VMM

I'm not quite sure. Exposing HT info to guest is only part of the story (but
absolutely required)

HT is a tech to share compute resources provided by a single physical core
so as to achieve better utilization.
OS should schedule two complementary instead of competing threads to one
physical core to achieve better performance.
Good example includes INT thread + FP thread, calc thread + mem bounded
thread etc.
If two threads contents for same resources in a physical core, the overall
throughput for one physical core may even drop as compare to the case of
only one thread.
I'm not sure how OS scheduler obtain such info, but maybe there are
instruction statistics for this purpose.

So it appears to me that only when the OS have full control over the core,
can it schedule best for HT.
(but this really depends on what kind of statistics is available. The
statement above is what I can guess best.)

In virtualized env, the above assumption can not be easily achieved.
People may be able to manipulate the CPU affinity through cpuset etc.
On the other hand, without human invention, I'm not sure if the xen
scheduler is doing anything special to handle HT.

On Thu, Jan 10, 2013 at 9:00 PM, ZHANG Zhi <zhizhang@xxxxxxxxxx> wrote:
>         Yeah, I think you're right.
>         Maybe it also depends on the virtualization techniques. If it's
hardware assisted virtualization, guest OS may detect the HT info, because
it can also scan the BIOS memory, right?
>
>
> re: Re: [Xen-devel] multi-core VMM
>
> Well, I can't agree.
> HW provide features and SW use it. Otherwise, HW feature is just a vain.
>
> PS: please keep the list CCed so that people are aware of the discussion.
Probably they have something to share on this topic.
>
> On Wed, Jan 9, 2013 at 1:37 PM, ZHANG Zhi <zhizhang@xxxxxxxxxx> wrote:
>> I think it's the hardware's business, namely the Intel or AMD, not the
hyperviosr's business. Xen does not have to consider such an issue.
>>
>> Re:
>> I think that's possible, you can check the cpupool related command in the
xl manpage.
>>
>> But I have a similar question -- In CPU like i7, 8 logical cores are not
fully equivalent.
>> They are hyper-threading over CMP. The two HT cores from a single
physical core are contending for pipeline resource. In linux kernel, there
is specific scheduler optimization for this case.
>> My question is that how XEN hypervisor handle this? Will such HT info be
available to the VM?
>> I guess there may be some different depend on whether you bind VCPU to
host core or not.
>>
>> Also, is there any different for dom0 && domU in this aspect?
>>
>> Thanks,
>> Timothy
>>
>> On Wed, Dec 19, 2012 at 2:20 PM, ZHANG Zhi <zhizhang@xxxxxxxxxx> wrote:
>>> Hi, list,
>>>
>>> A VMM provides a VMCS for each VM. How does the VMM assign system 
>>> resources for each VM? For example, in a multi-core environment, how 
>>> can I enable the VMM to run, say, on a two-core intel's processor 
>>> while I am able to force a VM to execute only on a specific core 
>>> upon initializing the Guest OS ? that is to say, how I can assure 
>>> the VM to believe that there is only one physical core running on this
machine?
>>>
>>> What should I do? Is it possible to assign a core when VMM starts a 
>>> new VM after the VMM is booted ?
>>>
>>>    Thanks a lot!
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@xxxxxxxxxxxxx
>>> http://lists.xen.org/xen-devel
>>>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel