[Xen-devel] Fwd: [PATCH] RFC: initial libxl support for xenpaging

[George Dunlap <George.Dunlap@xxxxxxxxxxxxx>]

Once again, didn't reply-to-all.  (Hmm, I'm pretty sure I actually did...)
 -George

---------- Forwarded message ----------
From: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
Date: Tue, Feb 28, 2012 at 3:43 PM
Subject: Re: [Xen-devel] [PATCH] RFC: initial libxl support for xenpaging
To: andres@xxxxxxxxxxxxxxxx


On Tue, Feb 28, 2012 at 3:25 PM, Andres Lagar-Cavilla
<andres@xxxxxxxxxxxxxxxx> wrote:
>> On Mon, Feb 27, 2012 at 2:45 PM, Andres Lagar-Cavilla
>> <andres@xxxxxxxxxxxxxxxx> wrote:
>>>> On Fri, 2012-02-24 at 17:12 +0000, Andres Lagar-Cavilla wrote:
>>>>> Sharing + balloon is mostly a bad idea. It's not forbidden or broken
>>>>> from the hypervisor angle, but it's a lose-lose game. Ballooning a
>>>>> shared page gains you nothing. The ballooner can't know what is shared
>>>>> and what isn't, in order to make an educated decision. And the sharer
>>>>> can't foresee what the balloon will victimize.
>>>>
>>>> Doesn't ballooning generally evict unused pages, because it allocates
>>>> new _free_ pages, even if they were shared is there any benefit to
>>>> doing
>>>> so?
>>>
>>> Certainly the balloon will pick free pages. The sharing daemon should
>>> not
>>> have shared those, but it's not unlikely that it will have.
>>>
>>> It's a classic semantic gap problem, and we were discussing this in the
>>> context of paging ("the pager should not have paged out page table
>>> pages")
>>>
>>> Beyond free pages, a prime target for sharing are read-only disk buffers
>>> in the page cache. Those are victim #2 for the balloon.
>>
>> Not exactly -- victim #2 would be read-only disk buffers *which have
>> not been read recently*.
>
> Everything's been read recently in Windows. Seriously ;)

Uum, do you really mean to say that Windows doesn't use the accessed
bit to try to find which pages are in active use?  That's rather
surprising.

>> Buffers which are in active use will not be
>> evicted.  So although evicting these pages from the guests' cache
>> doesn't buy the system any more memory, it doesn't have a major impact
>> on the guest either.
>
> That's debatable. Maybe guests shouldn't have a page cache then. Or a
> really small one.
>
> I'm not saying you're wrong, I'm saying that the answer is, as with many
> things, "it depends"

Sure, it depends. If there's lots of free memory, then sure, keep the
stuff around; maybe it will come in handy.  If memory is tight, then
the ideal is to have the pages which are actually used frequently --
whether they're in the page cache or elsewhere -- kept in memory, and
the rest paged out to disk / handed back to Xen.

>> In any case, if the guest experiences its own internal memory
>> pressure, these pages will be the first to go anyway.  After that, it
>> will go for trying to evict infrequently-used read-write pages --
>> which, if the pager is active, will already have been paged out to
>> disk; thus we'll end up with the double-paging problem.  This will
>> have a much larger impact on performance than uselessly evicting
>> little-used read-only pages.
>>
>> So I think that even though sharing+balloon will lead to some
>> occasional sub-optimal behavior, it's still a lot better than
>> sharing+paging and no ballooning.  Remember that ballooning was a
>> technique introduced in the paper by VMWare that talked about page
>> sharing -- they obviously thought sharing+ballooning was better than
>> sharing+paging.
>
> Ties in with Dan's thread. Depends on how much effort you spend choosing
> sharing and paging candidates, and how hard you inflate the balloon. I'm
> not in favor of any such overarching statement (even though I made one
> myself!!).

Yes, but at some point, if we want people to use this stuff, we have
to have a simple answer for xl, and recommendations for more
complicated answers.  I think re the "simple answer" possibilities for
xl and sharing, I think we have:
* Take whatever advantage of sharing we can; if we need to reduce
guest footprint (i.e., actual memory used), use ballooning first,
paging only afterwards
* Take whatever advantage of sharing we can; if we need to reduce
guest footprint, use only paging.

The first, as you say will have the disadvantage that early ballooning
will reduce sharing without getting memory back, but later ballooning
will evict pages properly without risking double paging.  The second
has the advantage that early paging will keep shared pages longer, but
the disadvantage that when the guest starts paging, you'll run into
the double paging problem.  As a cost-benefits analysis, I think the
second comes out worse.

(Of course, someone running some actual numbers would be the final
answer for this one, but I'm not sure anyone has the time or
inclination to do that at this point.)

Now, a really clever toolstack could use paging first, and then switch
to ballooning if it detects significant double-paging.  But I think
that's more than we want from xl.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel