Re: [Xen-devel] Proposed XENMEM_claim_pages hypercall: Analysis of problem and alternate solutions

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Tue, 2013-01-08 at 19:41 +0000, Dan Magenheimer wrote:
> Then a second premise that I would like to check to ensure we
> agree:  In the Oracle model, as I said, "open source guest kernels
> can intelligently participate in optimizing their own memory usage...
> such guests are now shipping" (FYI Fedora, Ubuntu, and Oracle Linux).
> With these mechanisms, there is direct guest->hypervisor interaction
> that, without knowledge of the toolstack, causes d->tot_pages
> to increase.  This interaction may (and does) occur from several
> domains simultaneously and the increase for any domain may occur
> frequently, unpredictably and sometimes dramatically.

Agreed.

> Ian, do you agree with this premise and that a "capacity allocation
> solution" (whether hypervisor-based or toolstack-based) must work
> properly in this context?

> Or are you maybe proposing to eliminate all such interactions?

I think these interactions are fine. They are obviously a key part of
your model. My intention is to suggest a possible userspace solution to
the claim proposal which continues to allow this behaviour.

> Or are you maybe proposing to insert the toolstack in the middle of
> all such interactions?

Not at all.

> Next, in your most recent reply, I think you skipped replying to my
> comment of "[in your proposal] the toolstack must make intelligent
> policy decisions about how to vary current_maxmem relative to
> lifetime_maxmem, across all the domains on the system [1]".  We
> seem to disagree on whether this need only be done twice per domain
> launch (once at domain creation start and once at domain creation
> finish, in your proposal) vs. more frequently.  But in either case,
> do you agree that the toolstack is not equipped to make policy
> decisions across multiple guests to do this

No, I don't agree.

> and that poor choices may have dire consequences (swapstorm, OOM) on a
> guest?

Setting maxmem on a domain does not immediately force a domain to that
amount of RAM and so the act of doing setting maxmem is not going to
cause a swap storm. (I think this relates to the "distinction between
current_maxmem and lifetime_maxmem was added for Citrix DMC support"
patch you were referring too below, previously to that Xen would reject
attempts to set max < current)

Setting maxmem doesn't even ask the domain to try and head for that
limit (that is the target which is a separate thing). So the domain
won't react to setting maxmem at all and unless it goes specifically
looking I don't think it would even be aware that its maximum has been
temporarily reduced.

Having set all the maxmem's on the domains you would then immediately
check if each domain has tot_pages under or over the temporary maxmem
limit. If all domains are under then the claim has succeeded and you may
proceed to build the domain. If any one domain is over then the claim
has failed and you need to reset all the maxmems back to the lifetime
value and try again on another host (I understand that this is an
accepted possibility with the h/v based claim approach too).

I forgot to say but you'd obviously want to use whatever controls tmem
provides to ensure it doesn't just gobble up the M bytes needed for the
new domain. It can of course continue to operate as normal on the
remainder of the spare RAM.

> AFAIK, the distinction between current_maxmem
> and lifetime_maxmem was added for Citrix DMC support.

As I mentioned above I think you are thinking of the patch which cause
the XEN_DOMCTL_max_mem hypercall to succeed even if tot_pages is
currently greater than the new requested maximum.

It's not quite the same thing as a distinction between current_maxmem
and lifetime_maxmem.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel