Re: [Xen-devel] Proposed new "memory capacity claim" hypercall/feature

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 30.10.12 at 18:13, Dan Magenheimer <dan.magenheimer@xxxxxxxxxx> wrote:
>>  From: Jan Beulich [mailto:JBeulich@xxxxxxxx]
>> With tmem being the odd one here, wouldn't it make more sense
>> to force it into no-alloc mode (apparently not exactly the same as
>> freezing all pools) for the (infrequent?) time periods of domain
>> creation, thus not allowing the amount of free memory to drop
>> unexpectedly? Tmem could, during these time periods, still itself
>> internally recycle pages (e.g. fulfill a persistent put by discarding
>> an ephemeral page).
> 
> Freeze has some unattractive issues that "claim" would solve
> (see below) and freeze (whether ephemeral pages are used or not)
> blocks allocations due to tmem, but doesn't block allocations due
> to selfballooning (or manual ballooning attempts by a guest user
> with root access).  I suppose the tmem freeze implementation could
> be extended to also block all non-domain-creation ballooning
> attempts but I'm not sure if that's what you are proposing.
> 
> To digress for a moment first, the original problem exists both in
> non-tmem systems AND tmem systems.  It has been seen in the wild on
> non-tmem systems.  I am involved with proposing a solution primarily
> because, if the solution is designed correctly, it _also_ solves a
> tmem problem.  (And as long as we have digressed, I believe it _also_
> solves a page-sharing problem on non-tmem systems.)  That said,
> here's the unattractive tmem freeze/thaw issue, first with
> the existing freeze implementation.
> 
> Suppose you have a huge 256GB machine and you have already launched
> a 64GB tmem guest "A".  The guest is idle for now, so slowly
> selfballoons down to maybe 4GB.  You start to launch another 64GB
> guest "B" which, as we know, is going to take some time to complete.
> In the middle of launching "B", "A" suddenly gets very active and
> needs to balloon up as quickly as possible or it can't balloon fast
> enough (or at all if "frozen" as suggested) so starts swapping (and,
> thanks to Linux frontswap, the swapping tries to go to hypervisor/tmem
> memory).  But ballooning and tmem are both blocked and so the
> guest swaps its poor little butt off even though there's >100GB
> of free physical memory available.

That's only one side of the overcommit situation you're striving
to get work right here: That same self ballooning guest, after
sufficiently more guest got started so that the rest of the memory
got absorbed by them would suffer the very same problems in
the described situation, so it has to be prepared for this case
anyway.

As long as the allocation times can get brought down to an
acceptable level, I continue to not see a need for the extra
"claim" approach you're proposing. So working on that one (or
showing that without unreasonable effort this cannot be
further improved) would be a higher priority thing from my pov
(without anyone arguing about its usefulness).

But yes, with all the factors you mention brought in, there is
certainly some improvement needed (whether your "claim"
proposal is a the right thing is another question, not to mention
that I currently don't see how this would get implemented in
a consistent way taking several orders of magnitude less time
to carry out).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel