[Xen-devel] RE: [PATCH v3] [linux-3.0+ for xen] tmem: self-ballooning and frontswap-selfshrinking

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> > > Can this be used by KVM or HyperV code? Can it be made generic? If so, 
> > > why not?
> > > If only Xen can use this, what would it entail for other balloon drivers
> > > to use this? Or is it that they really don't need to b/c none of them
> > > use the clean cache code?
> >
> > I'm not an expert on KVM nor on HyperV.  If either becomes capable
> > of supporting tmem and if the KVM/HyperV equivalent of balloon drivers
> > are suitable, concepts similar to selfballooning and frontswap-selfshrinking
> > are likely useful, though it would require quite a bit more study to
> > try to guess at how one might make the proposed code generic.
> >
> > I expect to talk to folk at the upcoming KVM Forum to consider
> > next steps, and another proposed patch (https://lkml.org/lkml/2011/6/30/354)
> > moves the in-kernel tmem support one step closer to supporting KVM.
> >
> > It's been awhile since I've communicated with Ky about tmem so I will
> > re-open that conversation.
> >
> > But it will probably be months/years before generic'izing this proposed
> > patch is feasible.  Seems best to cross that bridge later.
> 
> The mechanism this patch employs to "balloon" is quite generic - it uses the
> shrinker API. I am trying to understand from a technical perspective why this
> code cannot be outside Xen as generic code?

No it doesn't use the kernel shrinker API.  I added a couple of comments
to explain that (since you had assumed that in your earlier review).  The
kernel shrinker API calls shrinker function hooks in other subsystems to
request they surrender otherwise unused memory so that the kernel can
reclaim space when under memory pressure.  The frontswap shrinker causes
dirty anonymous pages held in transcendent memory to be forced back into
real kernel RAM when the kernel is NOT under memory pressure.

Both are doing shrinking, by the English definition, just shrinking
very different things with very different objectives which requires
very different interfaces and mechanisms.  Perhaps overloading the
term "shrinker" is confusing, but I think I've already coined enough
new words for tmem, don't you? ;-)

And, for completeness, self-ballooning is also very different than the
kernel shrinker API... a similar concept might be implemented using the
kernel shrinker API (and Daniel Kiper and I discussed that once on-list),
but the kernel "policy" for calling shrinker functions is a bit
obscure, has very different objectives, and would only work "one direction"
(reclaiming memory from the balloon driver, not providing it).

Yes, the core concept of using control theory to manage memory supply
vs. demand is potentially generic and eventually other tmem-capable
or tmem-ish environments might want to do something similar.  But
I do think that's likely to be months/years away and trying to generic'ize
the proposed code (and especially getting it upstream under these
circumstances) will likely be an exercise in frustration until other
in-kernel users try to use it first.  (The code implementing the core
concept is extremely small in any case... the vast majority of the code
in this patch is sysfs, initialization, and work management overhead,
which is unlikely to be generic anyway.)

Just my opinion...

Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel