Re: [Xen-devel] [PATCH 3/6] xen: credit1: increase efficiency and scalability of load balancing.

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

On Thu, 2017-03-02 at 11:06 +0000, Andrew Cooper wrote:
> On 02/03/17 10:38, Dario Faggioli wrote:
> > signed-off-by: Dario Faggioli <dario.faggioli@xxxxxxxxxx>
> > ---
> > Cc: George Dunlap <george.dunlap@xxxxxxxxxxxxx>
> > Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
>  
> Malcolm’s solution to this problem is https://github.com/xenserver/xe
> n-4.7.pg/commit/0f830b9f229fa6472accc9630ad16cfa42258966  This has
> been in 2 releases of XenServer now, and has a very visible
> improvement for aggregate multi-queue multi-vm intrahost network
> performance (although I can't find the numbers right now).
> 
Yep, as you know, I had become aware of that.

> The root of the performance problems is that pcpu_schedule_trylock()
> is expensive even for the local case, while cross-cpu locking is much
> worse.  Locking every single pcpu in turn is terribly expensive, in
> terms of hot cacheline pingpong, and the lock is frequently
> contended.
> 
> As a first opinion of this patch, you are adding another cpumask
> which is going to play hot cacheline pingpong.
> 
Can you clarify? Inside csched_load_balance(), I'm actually trading one
currently existing cpumask_and() with another.

Sure this new mask needs updating, but that only happens when a pCPUs
acquires or leaves the "overloaded" status.

Malcom's patch uses an atomic counter which does not fit very well in
Credit1's load balancer's logic, and in fact it (potentially) requires
an additional cpumask_cycle(). And it also comes with cache coherency
implications, doesn't it?

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel