Re: [Xen-devel] [Hackathon minutes] PV frontends/backends and NUMA machines

[Dario Faggioli <raistlin@xxxxxxxx>]

On mar, 2013-05-21 at 10:53 +0100, George Dunlap wrote:
> On Tue, May 21, 2013 at 10:24 AM, Wei Liu <wei.liu2@xxxxxxxxxx> wrote:
> > So the core thing in netback is almost ready, I trust Linux scheduler
> > now and don't pin kthread at all but relevant code shuold be easy to
> > add. I just checked my code, all memory allocation is already node
> > awared.
> >
> > As for the toolstack part, I'm not sure writing the initial node to
> > xenstore will be sufficient. Do we do inter-node migration? If so
> > frontend / backend should also update xenstore information as it
> > migrates?
> 
> We can of course migrate the vcpus, but migrating the actual memory
> from one node to another is pretty tricky, particularly for PV guests.
>  It won't be something that happens very often; when it does, we will
> need to sort out migrating the backend threads.
> 
Indeed.

> > IIRC the memory of a guest is striped through nodes, if it is this case,
> > how can pinning benefit? (I might be talking crap as I don't know much
> > about NUMA and its current status in Xen)
> 
> It's striped across nodes *of its NUMA affinity*.  So if you have a
> 4-node box, and you set its NUMA affinity to node 3, then the
> allocator will try to get all of the memory from node 3.  If its
> affinity is set to {2,3}, then the allocator will stripe it across
> nodes 2 and 3.
> 
Right. And other than that, the whole point of work items 1, 2 and 3 (in
George's list, at the beginning of this thread) is to make this striping
even "wiser". So, not only 'memory comes from nodes {2,3}' but '_this_
memory comes from node {2} and _that_ memory comes from {3}'. That's why
we think pinning would do it, but you're right (Wei), that is not true
right now, it will only be when we'll get those work items done. :-)

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel