Re: [Xen-devel] About vcpu wakeup and runq tickling in credit

[Dario Faggioli <raistlin@xxxxxxxx>]

On Fri, 2012-11-16 at 11:53 +0100, Dario Faggioli wrote:
> On Thu, 2012-11-15 at 12:18 +0000, George Dunlap wrote:
> > Maybe what we should do is do the wake-up based on who is likely to run 
> > on the current cpu: i.e., if "current" is likely to be pre-empted, look 
> > at idlers based on "current"'s mask; if "new" is likely to be put on the 
> > queue, look at idlers based on "new"'s mask.
> > 
> Ok, find attached the two (trivial) patches that I produced and am
> testing in these days. Unfortunately, early results shows that I/we
> might be missing something.
> 
I'm just came to thinking that this approach, although more, say,
correct, could have a bad impact on caches and locality in general.

In fact, suppose a new vcpu N wakes up on pcpu #x where another vcpu C
is running, with prio(N)>prio(C).

What upstream does is asking to #x and to all the idlers that can
execute N to reschedule. Doing both is, I think, wrong, as there's the
chance of ending up with N being scheduled on #x and C being runnable
but not running (in #x's runqueue) even if there are idle cpus that
could run it, as they're not poked (as already and repeatedly said).

What the patches do, in this case (remember (prio(N)>prio(C)), is asking
#x and all the idlers that can run C to reschedule, the effect being
that N will likely run on #x, after a context switch, and C will run
somewhere else, after a migration, potentially wasting its cache-hotness
(it is running after all!).

It looks like we can do better... Something like the below:
 + if there are no idlers where N can run, ask #x and the idlers where 
   C can run to reschedule (exactly what the patches do, although, they 
   do that _unconditionally_), as there isn't anything else we can do
   to try to make sure they both will run;
 + if *there*are* idlers where N can run, _do_not_ ask #x to reschedule 
   and only poke them to come pick N up. In fact, in this case, it is 
   not necessary to send C away for having both the vcpus ruunning, and 
   it seems better to have N experience the migration as, since it's 
   waking-up, it's more likely for him than for C to be cache-cold.

I'll run the benchmarks with this variant as soon as the one that I'm
running right now finish... In the meanwhile, any thoughts?

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel