Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Tue, 21 Feb 2017, Dario Faggioli wrote:
> On Tue, 2017-02-21 at 12:30 +0000, Julien Grall wrote:
> > On 21/02/2017 09:09, Dario Faggioli wrote:
> > > Well, TBH, we still are not entirely sure who the culprit is for
> > > high
> > > latency. There are spikes in Credit2, and I'm investigating that.
> > > But
> > > apart from them? I think we need other numbers with which we can
> > > compare the numbers that Stefano has collected.
> > 
> > I think the problem is because we save/restore the vCPU state when 
> > switching to the idle vCPU.
> > 
> That may well be. Or at least, that may well be part of the problem. I
> don't know enough of ARM to know whether it's the predominant cause of
> high latencies or not.
> 
> On x86, on Linux, polling is used to prevent the CPU to go in deep
> C-states. I was assuming things to be similar on ARM, and that's the
> reason why I thought introducing a polling mode could have been useful
> (although wasteful).
> 
> But you guys are the ones that know whether or not that is the case
> (and in another email, you seem to say it's not).

The total cost of context switching is about 1100ns, which is
significant.  The remaining 1300ns difference between vwfi=sleep and
vwfi=idle is due to sched_op(wait) and sched_op(schedule).

As I mentioned, I have a patch to zero the context switch time when we
can back and force between a regular vcpu and the idle vcpu, but it's a
complex ugly patch I don't know if we want to maintain.


> > Let say the only 1 vCPU can run on the pCPU, when the vCPU is issuing
> > a 
> > WFI the following steps will happen:
> >       * WFI trapped and vcpu blocked
> >       * save vCPU state
> >       * run idle_loop
> > -> Interrupt incoming for the guest
> >       * restore vCPU state
> >       * back to the guest
> > 
> > Saving/restoring on ARM requires to context switch all the state of
> > the 
> > VM (this is not saved in memory when entering in the hypervisor).
> > This 
> > include things like system register, interrupt controller state,
> > FPU...
> > 
> Yes. In fact, on x86, we have what we call 'lazy context switch', which
> deals specifically with some aspect of this situation.
> 
> Indeed it seems like implementing the same on ARM --if you don't have
> it already, and if possible-- would be useful in this case too.

This sounds like what I have.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel