[Xen-devel] [PATCH] xend: Fix non-contiguous NUMA node assignment

[Andre Przywara <andre.przywara@xxxxxxx>]

Hi,

it seems that I missed a point in this whole addition of max_node_id. I 
see the difference in the Xen HV part, so nr_nodes got replaced with 
max_node_id in physinfo_t (and xc_physinfo_t, respectively).
But where does this value help in xend? There is no single Python 
reference to the physinfo()'s max_node_id field, instead all functions 
use the old (but now bogus) nr_nodes variable.
So in the attached patch I kept the xc.physinfo() returned dictionary 
with only a nr_nodes field, calculated by simply adding 1 to max_node_id 
from libxc. Empty nodes can (and will) be detected by iterating through 
the node_to_cpus and node_to_memory lists.
Nodes without memory should not be considered during guest's memory 
allocation, but will be used for further CPU affinity setting if the 
number of VCPUs exceeds the number of cores per node.
Please correct me if I am totally wrong on this, but this seems to work 
much better in my case.
Regards,
Andre.

Signed-off-by: Andre Przywara <andre.przywara@xxxxxxx>

--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448 3567 12
----to satisfy European Law for business letters:
Advanced Micro Devices GmbH
Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen
Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni
Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

diff -r db8a882f5515 tools/python/xen/lowlevel/xc/xc.c
--- a/tools/python/xen/lowlevel/xc/xc.c Thu Jan 14 14:11:25 2010 +0000
+++ b/tools/python/xen/lowlevel/xc/xc.c Fri Jan 15 14:20:05 2010 +0100
@@ -1078,7 +1078,7 @@
 #define MAX_CPU_ID 255
     xc_physinfo_t info;
     char cpu_cap[128], virt_caps[128], *p;
-    int i, j, max_cpu_id, nr_nodes = 0;
+    int i, j, max_cpu_id;
     uint64_t free_heap;
     PyObject *ret_obj, *node_to_cpu_obj, *node_to_memory_obj;
     PyObject *node_to_dma32_mem_obj;
@@ -1115,7 +1115,6 @@
     node_to_dma32_mem_obj = PyList_New(0);
     for ( i = 0; i <= info.max_node_id; i++ )
     {
-        int node_exists = 0;
         PyObject *pyint;
 
         /* CPUs. */
@@ -1127,14 +1126,12 @@
             pyint = PyInt_FromLong(j);
             PyList_Append(cpus, pyint);
             Py_DECREF(pyint);
-            node_exists = 1;
         }
         PyList_Append(node_to_cpu_obj, cpus); 
         Py_DECREF(cpus);
 
         /* Memory. */
         xc_availheap(self->xc_handle, 0, 0, i, &free_heap);
-        node_exists = node_exists || (free_heap != 0);
         pyint = PyInt_FromLong(free_heap / 1024);
         PyList_Append(node_to_memory_obj, pyint);
         Py_DECREF(pyint);
@@ -1145,13 +1142,10 @@
         PyList_Append(node_to_dma32_mem_obj, pyint);
         Py_DECREF(pyint);
 
-        if ( node_exists )
-            nr_nodes++;
     }
 
-    ret_obj = 
Py_BuildValue("{s:i,s:i,s:i,s:i,s:i,s:i,s:l,s:l,s:l,s:i,s:s:s:s}",
-                            "nr_nodes",         nr_nodes,
-                            "max_node_id",      info.max_node_id,
+    ret_obj = Py_BuildValue("{s:i,s:i,s:i,s:i,s:i,s:l,s:l,s:l,s:i,s:s,s:s}",
+                            "nr_nodes",         info.max_node_id + 1,
                             "max_cpu_id",       info.max_cpu_id,
                             "threads_per_core", info.threads_per_core,
                             "cores_per_socket", info.cores_per_socket,

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel