Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Mon, 20 Feb 2017, Dario Faggioli wrote:
> On Mon, 2017-02-20 at 19:38 +0000, Julien Grall wrote:
> > On 20/02/17 19:20, Dario Faggioli wrote:
> > > E.g., if vCPU x of domain A wants to go idle with a WFI/WFE, but
> > > the
> > > host is overbooked and currently really busy, Xen wants to run some
> > > other vCPU (of either the same of another domain).
> > > 
> > > That's actually the whole point of virtualization, and the reason
> > > why
> > > overbooking an host with more vCPUs (from multiple guests) than it
> > > has
> > > pCPUs works at all. If we start letting guests put the host's pCPUs
> > > to
> > > sleep, not only the scheduler, but many things would break, IMO!
> > 
> > I am not speaking about general case but when you get 1 vCPU pinned
> > to 1 
> > pCPU (I think this is Stefano use case). No other vCPU will run on
> > this 
> > pCPU. So it would be fine to let the guest do the WFI.
> > 
> Mmm... ok, yes, in that case, it may make sense and work, from a, let's
> say, purely functional perspective. But still I struggle to place this
> in a bigger picture.

I feel the same way as you, Dario. That said, if we could make it work
without breaking too many assumptions in Xen, it would be a great
improvement for this use-case.


> For instance, as you say, executing a WFI from a guest directly on
> hardware, only makes sense if we have 1:1 static pinning. Which means
> it can't just be done by default, or with a boot parameter, because we
> need to check and enforce that there's only 1:1 pinning around.

That's right, but we don't have a way to recognize or enforce 1:1 static
pinning at the moment, do we? But maybe the nop scheduler we discussed
could be a step in the right direction.


> Is it possible to decide whether to trap and emulate WFI, or just
> execute it, online, and change such decision dynamically? And even if
> yes, how would the whole thing work? When the direct execution is
> enabled for a domain we automatically enforce 1:1 pinning for that
> domain, and kick all the other domain out of its pcpus? What if they
> have their own pinning, what if they also have 'direct WFI' behavior
> enabled?

Right, I asked myself those questions as well. That is why I wrote "it
breaks the scheduler" in the previous email. I don't think it can work
today, but it could work one day, building on top of the nop scheduler.


> If it is not possible to change all this online and on a per-domain
> basis, what do we do? When dooted with the 'direct WFI' flag, we only
> accept 1:1 pinning? Who should enforce that, the setvcpuaffinity
> hypercall?
> 
> These are just examples, my point being that in theory, if we consider
> a very specific usecase or set of usecase, there's a lot we can do. But
> when you say "why don't you let the guest directly execute WFI", in
> response to a patch and a discussion like this, people may think that
> you are actually proposing doing it as a solution, which is not
> possible without figuring out all the open questions above (actually,
> probably, more) and without introducing a lot of cross-subsystem
> policing inside Xen, which is often something we don't want.

+1


> But, if you let me say this again, it looks to me we are trying to
> solve too many problem all at once in this thread, should we try
> slowing down/refocusing? :-)

Indeed. I think this patch can improve some use-cases with little
maintenance cost. It's pretty straightforward to me.


> > But in 
> > that case, using WFI in the guest may not have been the right things
> > to do.
> > 
> But if the guest is, let's say, Linux, does it use WFI or not? And is
> it the right thing or not?
> 
> Again, the fact you're saying this probably means there's something I
> am either missing or ignoring about ARM.

Linux uses WFI


> > I have heard use case where people wants to disable the scheduler
> > (e.g a 
> > nop scheduler) because they know only 1 vCPU will ever run on the
> > pCPU. 
> > This is exactly the use case I am thinking about.
> > 
> Sure! Except that, in Xen, we don't know whether we have, and always
> will, 1 vCPU ever run on each pCPU. Nor we have a way to enforce that,
> neither in toolstack nor in the hypervisor. :-P

Exactly! I should have written it more clearly from the beginning.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel