Re: [Xen-devel] Proposed new "memory capacity claim" hypercall/feature

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> From: Keir Fraser [mailto:keir.xen@xxxxxxxxx]
> Subject: Re: Proposed new "memory capacity claim" hypercall/feature
> 
> On 29/10/2012 18:06, "Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx> wrote:
> 
> > The objective of the design is to ensure that a multi-threaded
> > toolstack can atomically claim a specific amount of RAM capacity for a
> > domain, especially in the presence of independent dynamic memory demand
> > (such as tmem and selfballooning) which the toolstack is not able to track.
> > "Claim X 50G" means that, on completion of the call, either (A) 50G of
> > capacity has been claimed for use by domain X and the call returns
> > success or (B) the call returns failure.  Note that in the above,
> > "claim" explicitly does NOT mean that specific physical RAM pages have
> > been assigned, only that the 50G of RAM capacity is not available either
> > to a subsequent "claim" or for most[2] independent dynamic memory demands.
> 
> I don't really understand the problem it solves, to be honest. Why would you
> not just allocate the RAM pages, rather than merely making that amount of
> memory unallocatable for any other purpose?

Hi Keir --

Thanks for the response!

Sorry, I guess the answer to your question is buried in the
thread referenced (as [1]) plus a vague mention in this proposal.

The core issue is that, in the hypervisor, every current method of
"allocating RAM" is slow enough that if you want to allocate millions
of pages (e.g. for a large domain), the total RAM can't be allocated
atomically.  In fact, it may even take minutes, so currently a large
allocation is explicitly preemptible, not atomic.

The problems the proposal solves are (1) some toolstacks (including
Oracle's "cloud orchestration layer") want to launch domains in parallel;
currently xl/xapi require launches to be serialized which isn't very
scalable in a large data center; and (2) tmem and/or other dynamic
memory mechanisms may be asynchronously absorbing small-but-significant
portions of RAM for other purposes during an attempted domain launch.
In either case, this is a classic race, and a large allocation may
unexpectedly fail, possibly even after several minutes, which is
unacceptable for a data center operator or for automated tools trying
to launch any very large domain.

Does that make sense?  I'm very open to other solutions, but the
only one I've heard so far was essentially "disallow independent
dynamic memory allocations" plus keep track of all "claiming" in the
toolstack.

Thanks,
Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel