[Xen-devel] Re: [RFC][PATCH] Scheduler interface changes for credit2

[George Dunlap <george.dunlap@xxxxxxxxxxxxx>]

Keir Fraser wrote:
> They're all fine, even probably sched-context_switch-callback.diff, which I
> suppose is your new alternative to having vcpus-which-arent-yet-schedulable
> left on the shared runqueue? I suppose, although I reckon it could still be
> done another way and using the vcpu_migrate logic, that this is a smaller
> and neater way to do it really.
>   
Exactly.  The migrate logic calls cpu_pick, which is also called from 
other contexts.  Having the same function try to do both, and setting 
the "migrating" flag when the vcpu isn't migrating, just seems hackish 
and likely to be confusing to me.  It does add another "Check to see if 
this function exists" on the hot path for the other schedulers.
I suppose we could think about changing things around, so that there's a 
generic "callback on context switch" flag.  Then we'd only have one 
callback in the context switch code, and the check would only happen if 
the flag was set (reducing cache footprint).  It would be a slightly 
larger patch, and make changes to the other schedulers, but those would 
be pretty minimal.
I'll play around with it and see how it looks.

-George
> All this are apply-able after 4.0.
>
>  -- Keir
>
> On 22/02/2010 15:22, "George Dunlap" <George.Dunlap@xxxxxxxxxxxxx> wrote:
>
>   
> > Keir, any thoughts?  Does this seem like a reasonable approach?
> >
> > Thanks,
> >  -George
> >
> > On Mon, Feb 15, 2010 at 5:20 PM, George Dunlap
> > <George.Dunlap@xxxxxxxxxxxxx> wrote:
> >     
> > > The two attached patches change the scheduler interface to allow
> > > credit2 to have several cpus share the same runqueue.  The patches
> > > should have almost no impact on the current schedulers.  The patches
> > > and their reasonings are below.  I've also attached the patches for
> > > the prototype credit2 scheduler, for reference.
> > >
> > > * Add a context swich callback (sched-context_switch-callback.diff)
> > >
> > > Add a callback to tell a scheduler that a vcpu has been completely
> > > context-switched off a cpu.
> > >
> > > When sharing a runqueue, we can't put a scheduled-out vcpu back on the
> > > runqueue until it's been completely de-scheduled, because it may be
> > > grabbed by another processor before it's ready.  This callback allows
> > > a scheduler to detect when a vcpu on its way out is completely off the
> > > processor, so that it can put the vcpu on the runqueue.
> > >
> > > * Allow sharing of locks between cpus (sched-spin_lock-pointers.diff)
> > >
> > > Have per-cpu pointers, initialized to per-cpu locks, which the
> > > scheduler may change during its init to reconfigure locking
> > > granularity.
> > >
> > > There are a number of race conditions having to do with updating of
> > > v->is_running and v->processor, all having to do with the fact that
> > > vcpus may change cpus without an explicit migrate.  Furthermore, the
> > > scheduler needs runqueues to be covered by a lock as well.  The
> > > cleanest way to solve all of these is to have the scheduler lock and
> > > the runqueue lock coincide.
> > >
> > > * Add a "scheduler" trace class (trace-sched-class.diff)
> > > Uses defined on a per-scheduler basis
> > >
> > > I've been running parallel kernel compiles on a 16-way box (2x4x2) for
> > > several hours now without deadlocks or BUG()s.  As far as I'm
> > > concerned, with these changes, credit2 is now ready to be checked in,
> > > as long as it's not set to the default scheduler.
> > >
> > > All of the above:
> > > Signed-off-by: George Dunlap <george.dunlap@xxxxxxxxxxxxx>
> > >
> > >       
>
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel