[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] NUMA TODO-list for xen-devel



On 08/03/2012 11:23 AM, Jan Beulich wrote:
On 02.08.12 at 18:36, George Dunlap <George.Dunlap@xxxxxxxxxxxxx> wrote:
On Thu, Aug 2, 2012 at 2:34 PM, Dario Faggioli <raistlin@xxxxxxxx> wrote:
On Thu, 2012-08-02 at 10:43 +0100, Jan Beulich wrote:
On 01.08.12 at 18:16, Dario Faggioli <raistlin@xxxxxxxx> wrote:
     - Virtual NUMA topology exposure to guests (a.k.a guest-numa). If a
       guest ends up on more than one nodes, make sure it knows it's
       running on a NUMA platform (smaller than the actual host, but
       still NUMA). This interacts with some of the above points:

The question is whether this is really useful beyond the (I would
suppose) relatively small set of cases where migration isn't
needed.

Mmm... Not sure I'm getting what you're saying here, sorry. Are you
suggesting that exposing a virtual topology is not a good idea as it
poses constraints/prevents live migration?

Honestly, what would be the problems with migration? NUMA awareness is actually a software optimization, so we will not really break something if the advertised topology isn't the real one. This is especially true if we lower the number of NUMA nodes. Say the guest starts with two nodes and then gets migrated to a machine where it can happily live in one node. There would be some extra effort by the guest OS to obey the virtual NUMA topology, but if there isn't actually a NUMA penalty anymore this shouldn't really hurt, right? Even if we would need to go to a machine where we have more nodes for a certain guest than before, this is actually what we have today: guest NUMA unawareness. I am not sure if this is really a migration showstopper, and certainly not a NUMA guest showstopper.

But we could make it a config file option, so we leave this decision to the admin. I have talked to people with huge guests, they keep asking me about this feature.


If yes, well, I mostly agree that this is an huge issue, and that's why
I think wee need some bright idea on how to deal with it. I mean, it's
easy to make it optional and let it automatically disable migration,
giving users the choice what they prefer, but I think this is more
dodging the problem than dealing with it! :-P

        * consider this during automatic placement for
          resuming/migrating domains (if they have a virtual topology,
          better not to change it);
        * consider this during memory migration (it can change the
          actual topology, should we update it on-line or disable memory
          migration?)

I think we could use cpu hot-plug to change the "virtual topology" of
VMs, couldn't we?  We could probably even do that on a running guest
if we really needed to.

Hmm, not sure - using hotplug behind the back of the guest might
be possible, but you'd first need to hot-unplug the vCPU. That's
something that I don't think you can do on HVM guests (and for
PV guests, guest visible NUMA support makes even less sense
than for HVM ones).

I don't think that hotplug would really work. I have checked this some times ago, at least the Linux NUMA code cannot be really fooled by this. The SRAT table is firmware defined and static by nature, so there is no code in Linux to change the NUMA topology at runtime. This is especially true for the memory layout.

But as said above, I don't really buy this as an argument against guest NUMA. At least provide it as an option to people who know what they do.

Regards,
Andre.


--
Andre Przywara
AMD-OSRC (Dresden)
Tel: x29712


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.