Re: [Xen-devel] [Hackathon minutes] PV frontends/backends and NUMA machines

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Tue, May 21, 2013 at 10:44:02AM +0200, Roger Pau Monné wrote:
[...]
> > 4. Teach the toolstack to pin the netback threads to dom0 vcpus
> > running on the correct node (s)
> > 
> > Dario will do #1.  I volunteered to take a stab at #2 and #3.  #4 we
> > should be able to do independently of 2 and 3 -- it should give a
> > slight performance improvement due to cache proximity even if dom0
> > memory is striped across the nodes.
> > 
> > Does someone want to volunteer to take a look at #4?  I suspect that
> > the core technical implementation will be simple, but getting a stable
> > design that everyone is happy with for the future will take a
> > significant number of iterations.  Learn from my fail w/ USB hot-plug
> > in 4.3, and start the design process early. :-)
> 
> #4 is easy to implement from my POV in blkback, you just need to write a
> node in the xenstore backend directory that tells blkback to pin the
> created kthread to a specific NUMA node, and make sure that the memory
> used for that blkback instance is allocated from inside the kthread. My
> indirect descriptors series already removes any shared structures
> between different blkback instances, so some part of this work is
> already done. And I guess that something similar could be implemented
> for QEMU/Qdisk from the toolstack level (pin the qemu process to a
> specific NUMA node).
> 
> I'm already quite familiar with the blkback code, so I can take care of
> #4 for blkback.
> 

So the core thing in netback is almost ready, I trust Linux scheduler
now and don't pin kthread at all but relevant code shuold be easy to
add. I just checked my code, all memory allocation is already node
awared.

As for the toolstack part, I'm not sure writing the initial node to
xenstore will be sufficient. Do we do inter-node migration? If so
frontend / backend should also update xenstore information as it
migrates?

IIRC the memory of a guest is striped through nodes, if it is this case,
how can pinning benefit? (I might be talking crap as I don't know much
about NUMA and its current status in Xen)


Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel