Re: [Xen-devel] Pointed questions re Xen memory overcommit

[George Dunlap <George.Dunlap@xxxxxxxxxxxxx>]

On Wed, Feb 29, 2012 at 5:26 PM, Dan Magenheimer
<dan.magenheimer@xxxxxxxxxx> wrote:
>> From: Olaf Hering [mailto:olaf@xxxxxxxxx]
>> Subject: Re: [Xen-devel] Pointed questions re Xen memory overcommit
>>
>> On Mon, Feb 27, Dan Magenheimer wrote:
>>
>> > > From: Olaf Hering [mailto:olaf@xxxxxxxxx]
>> > > To me memory overcommit means swapping, which is what xenpaging does:
>> > > turn the whole guest gfn range into some sort of virtual memory,
>> > > transparent to the guest.
>> > >
>> > > xenpaging is the red emergency knob to free some host memory without
>> > > caring about the actual memory constraints within the paged guests.
>> >
>> > Sure, but the whole point of increasing RAM in one or more guests
>> > is to increase performance, and if overcommitting *always* means
>> > swapping, why would anyone use it?
>>
>> The usage patterns depend on the goal. As I wrote, swapping frees host
>> memory so it can be used for something else, like starting yet another
>> guest on the host.
>
> OK, I suppose xenpaging by itself could be useful in a situation such as:
>
> 1) A failover occurs from machine A that has lots of RAM to machine B
>   that has much less RAM, and even horribly bad performance is better
>   than total service interruption.
> 2) All currently running VMs have been ballooned down "far enough"
>   and either have no swap device or insufficiently-sized swap devices,
>   Xen simply has no more free space, and horrible performance is
>   acceptable.
>
> The historical problem with "hypervisor-based-swapping" solutions such
> as xenpaging is that it is impossible to ensure that "horribly bad
> performance" doesn't start occurring under "normal" circumstances
> specifically because (as Tim indirectly concurs below), policies
> driven by heuristics and external inference (i.e. dom0 trying
> to guess how much memory every domU "needs") just don't work.
>
> As a result, VMware customers outside of some very specific domains
> (domains possibly overlapping with Snowflock?) will tell you
> that "memory overcommit sucks and so we turn it off".
>
> Which is why I raised the question "why are we doing this?"...
> If the answer is "Snowflock customers will benefit from it",
> that's fine.  If the answer is "to handle disastrous failover
> situations", that's fine too.  And I suppose if the answer is
> "so that VMware can't claim to have a feature that Xen doesn't
> have (even if almost nobody uses it)", I suppose that's fine too.
>
> I'm mostly curious because I've spent the last four years
> trying to solve this problem in a more intelligent way
> and am wondering if the "old way" has improved, or is
> still just the old way but mostly warmed over for Xen.
> And, admittedly, a bit jealous because there's apparently
> so much effort going into the "old way" and not toward
> "a better way".

Is it possible to use the "better way" in Windows?  If not, then those
who want to support Windows guests are stuck with the old way, until
MS discovers / "re-invents" tmem. :-)  (Maybe you should give a your
tmem talk at MS research?  Or try to give it again if you already
have?)  Even then there'd still be a market for supporting all those
pre-tmem OS's for quite a while.

Apart from that, here's my perspective:
* Having page sharing is good, because of potential memory savings in
VDI deployments, but also because of potential for fun things like VM
fork, &c
* Having page sharing requires you to have an emergency plan if
suddenly all the VMs write to their pages and un-share them.
Hypervisor paging is the only reasonable option here.

If it makes you feel any better, one of the suggested default policies
for "what to do with all the memory sharing generates" was "put it
into a tmem pool". :-)  That, and as we try to hammer out what the
implications of default policies are, the semantic gap between balloon
drivers, paging, and sharing, is rearing a very ugly head -- I would
much rather just hand the paging/ballooning stuff over to tmem.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel