Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Wed, 22 Feb 2017, Edgar E. Iglesias wrote:
> On Tue, Feb 21, 2017 at 07:20:29PM +0000, Julien Grall wrote:
> > 
> > 
> > On 21/02/2017 18:30, Stefano Stabellini wrote:
> > >On Tue, 21 Feb 2017, Julien Grall wrote:
> > >>Hi Stefano,
> > >>
> > >>On 21/02/17 17:49, Stefano Stabellini wrote:
> > >>>On Tue, 21 Feb 2017, Dario Faggioli wrote:
> > >>>>On Tue, 2017-02-21 at 13:46 +0000, George Dunlap wrote:
> > >>>>Oh, actually, if --which I only now realize may be what you are
> > >>>>referring to, since you're talking about "guest burning its credits"--
> > >>>>you let the vCPU put the pCPU to sleep *but*, when it wakes up (or when
> > >>>>the scheduler runs again for whatever reason), you charge to it for all
> > >>>>the time the the pCPU was actually idle/sleeping, well, that may
> > >>>>actually  not break scheduling, or cause disruption to the service of
> > >>>>other vCPUs.... But indeed I'd consider it rather counter intuitive a
> > >>>>behavior.
> > >>>
> > >>>How can this be safe? There could be no interrupts programmed to wake up
> > >>>the pcpu at all. In fact, I don't think today there would be any, unless
> > >>>we set one up in Xen for the specific purpose of interrupting the pcpu
> > >>>sleep.
> > >>>
> > >>>I don't know the inner working of the scheduler, but does it always send
> > >>>an interrupt to other pcpu to schedule something?
> > >>
> > >>You still seem to assume that WFI/WFE is the only way to get a vCPU
> > >>unscheduled. If that was the case it would be utterly wrong because you 
> > >>cannot
> > >>expect a guest to use them.
> > >>
> > >>>
> > >>>What if there are 2 vcpu pinned to the same pcpu? This cannot be fair.
> > >>
> > >>Why wouldn't it be fair? This is the same situation as a guest vCPU not 
> > >>using
> > >>WFI/WFE.
> > >
> > >I read your suggestion as trapping WFI in Xen, then, depending on
> > >settings, executing WFI in the Xen trap handler to idle the pcpu. That
> > >doesn't work. But I take you suggested not trapping wfi (remove
> > >HCR_TWI), executing the instruction in guest context. That is what we
> > >used to do in the early days (before a780f750). It should be safe and
> > >possibly even quick. I'll rerun the numbers and let you know.
> > 
> > My first suggestion was to emulate WFI in Xen, which I agree is not safe :).
> > 
> > I think not trapping WFI will have the best performance but may impact the
> > credit of the vCPU as mentioned by Dario and George.
> 
> I agree, wfi in guest context or at least with everything prepared to return 
> to
> the current guest would be great.

And the new numbers look good:

         AVG        MAX     WARM_MAX
credit1  1850           2650    1950
credit2  1850           2950    1840 

We are hitting the same levels as the non-WFI case. Nice!


> An option to enable this would work fine for our use-cases. Or if we could
> at runtime detect that it's the best approach given scheduling (i.e
> exclusive vCPU/pCPU pinning) even better.

The option is easy and we can do it today. We might be able to do
automatic enablement once we have a "nop" scheduler.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel