Re: [Xen-devel] NUMA TODO-list for xen-devel

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> From: Dario Faggioli [mailto:raistlin@xxxxxxxx]
> Subject: [Xen-devel] NUMA TODO-list for xen-devel
> 
> Hi everyone,

Hi Dario --

Thanks for your great work on NUMA... an interest area of
mine but one, sadly, I haven't been able to give much time to,
so I'm glad you've taken this bull by the horns.

I've been sitting on an idea for some time that probably
deserves some exposure on your list.  Naturally, it involves
my favorite topic tmem (readers, please don't tune out yet :-).

It has occurred to me that a fundamental tenet of NUMA
is to put infrequently used data on "other" nodes, while
pulling frequently used data onto a "local" node.

Tmem very nicely separates infrequently-used data from
frequently-used data with an API/ABI that is now fully
implemented in upstream Linux.

If Xen had a "alloc_page_on_any_node_but_the_current_one()"
(or "any_node_except_this_guests_node_set" for multinode guests)
and Xen's tmem implementation were to use it, especially
in combination with selfballooning (also upstream), this
could solve a significant part of the NUMA problem when running
tmem-enabled guests.  The most frequently used data
stays in the guest (thus in the guest's "current node")
and the less frequently used data lives in tmem in the
hypervisor (on the guest's "complement guest's

Naturally, this doesn't solve any NUMA problems at all for
tmem-ignorant or tmem-disabled guests, but if it works
sufficiently well for tmem-enabled guests, that might
encourage other OS's to do a simple implementation of tmem.

Sadly, I'm not able to invest much time in this idea,
but the combination of tmem and NUMA might interest some
developers and/or grad students, in which case I'd be happy
to spend a little time assisting.

I'll be at Xen Summit for at least the first day, so we
can chat more if you are interested.  George/Jan, I suspect
you have the best knowledge of tmem outside of Oracle as well
as being NUMA-fluent, so I'd appreciate your thoughts as well!

Thanks,
Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel