Re: [Xen-devel] Proposed XENMEM_claim_pages hypercall: Analysis of problem and alternate solutions

[Tim Deegan <tim@xxxxxxx>]

Hi, 

At 13:38 -0800 on 02 Jan (1357133898), Dan Magenheimer wrote:
> > The discussion ought to be around the actual problem, which is (as far
> > as I can see) that in a system where guests are ballooning without
> > limits, VM creation failure can happen after a long delay.  In
> > particular it is the delay that is the problem, rather than the failure.
> > Some solutions that have been proposed so far:
> >  - don't do that, it's silly (possibly true but not helpful);
> >  - this reservation hypercall, to pull the failure forward;
> >  - make allocation faster to avoid the delay (a good idea anyway,
> >    but can it be made fast enough?);
> >  - use max_pages or similar to stop other VMs using all of RAM.
> 
> Good summary.  So, would you agree that the solution selection
> comes down to: "Can max_pages or similar be used effectively to
> stop other VMs using all of RAM? If so, who is implementing that?
> Else the reservation hypercall is a good solution." ?

Not quite.  I think there are other viable options, and I don't
particularly like the reservation hypercall.

I can still see something like max_pages working well enough.  AFAICS
the main problem with that solution is something like this: because it
limits the guests individually rather than collectively, it prevents
memory transfers between VMs even if they wouldn't clash with the VM
being built.  That could be worked around with an upcall to a toolstack
agent that reshuffles things on a coarse granularity based on need.  I
agree that's slower than having the hypervisor make the decisions but
I'm not convinced it'd be unmanageable.

Or, how about actually moving towards a memory scheduler like you
suggested -- for example by integrating memory allocation more tightly
with tmem.  There could be an xsm-style hook in the allocator for
tmem-enabled domains.  That way tmem would have complete control over
all memory allocations for the guests under its control, and it could
implement a shared upper limit.  Potentially in future the tmem
interface could be extended to allow it to force guests to give back
more kinds of memory, so that it could try to enforce fairness (e.g. if
two VMs are busy, why should the one that spiked first get to keep all
the RAM?) or other nice scheduler-like properties.

Or, you could consider booting the new guest pre-ballooned so it doesn't
have to allocate all that memory in the build phase.  It would boot much
quicker (solving the delayed-failure problem), and join the scramble for
resources on an equal footing with its peers.

> > My own position remains that I can live with the reservation hypercall,
> > as long as it's properly done - including handling PV 32-bit and PV
> > superpage guests.
> 
> Tim, would you at least agree that "properly" is a red herring?

I'm not quite sure what you mean by that.  To the extent that this isn't
a criticism of the high-level reservation design, maybe.  But I stand by
it as a criticism of the current implementation.

Cheers,

Tim.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel