Re: [Xen-devel] arm64: Approach for DT based NUMA and issues

[Julien Grall <julien.grall@xxxxxxx>]

Hi Dario,

On 27/11/2016 01:01, Dario Faggioli wrote:
> On Sat, 2016-11-26 at 12:29 +0530, Vijay Kilari wrote:
> > 4) Generating memory nodes for DOM0
> > ---------------------------------------------------------
> > Linux kernel device drivers that uses devm_zalloc(), tries to
> > allocate memory
> > from local memory node. So Dom0 needs to have memory allocated on all
> > the
> > available nodes of the system.
> So, first of all, I know next to nothing about ARM, NUMA or non-NUMA.
>
> That being said, providing a guest with a NUMA layout is what we call
> "virtual NUMA" (vNUMA). It is implemented (although it still has some
> issues) for x86 already (so you can have a look), but only for DomU.
>
> I agree that we need to support vNUMA for Dom0 sooner rather than
> later, and I agree that Dom0 is a bit special, so some tricks may be
> necessary. But until we don't implement vNUMA for Dom0, Dom0 is just a
> non-NUMA virtual machine, and the kernel running inside that should
> just behave like it behaves on a non-NUMA box.
>
> Again, I don't know much about ARM, but I think that, until we don't
> have vNUMA for Dom0, that devm_zalloc() thing will just see 1 and only
> 1 NUMA node from which to allocate memory.
I would rather divide the NUMA work for ARM in 2 distinct tasks:
        - Make Xen NUMA-aware
        - Make DOM0 NUMA-aware

Vijay, if I understood correctly what Dario said, on x86 DOM0 is not yet 
NUMA-aware. Aside performance improvement, will there be any technical 
problem (such as PCI devices will not work) if we don't expose NUMA to 
DOM0 from the beginning?
> That being said, and for what it's worth...
>
> > Ex: SMMU driver of device on node 1 tries to allocate memory
> > on node 1.
> >
> > ISSUE:
> >  - Dom0's memory should be split across all the available memory
> > nodes
> >    of the system and memory nodes should be generated accordingly.
> ...This is the default behavior, at least on x86.
Are you speaking about the command line parameter dom0_nodes?

> >  - Memory DT node generated by Xen for Dom0 should populate numa-
> > node-id
> >    information.
> Generating DT nodes for Dom0 is exactly what I mean when I say
> "implementing / enabling vNUMA for Dom0" (in this case on ARM).
>
> So, yes, let's do it, but let's discuss how to do it properly (e.g., if
> there's anything that can be common between archs, such as some bits of
> the interface).
I would expect vNUMA for Dom0 to be common between x86 and ARM.

Regards,

--
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel