Re: [Xen-devel] [PATCH 1/5] x86: allow specifying the NUMA nodes Dom0 should run on

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

On Thu, 2015-02-26 at 13:52 +0000, Jan Beulich wrote:
> ... by introducing a "dom0_nodes" option augmenting the "dom0_mem" and
> "dom0_max_vcpus" ones.
> 
> Note that this gives meaning to MEMF_exact_node specified alone (i.e.
> implicitly combined with NUMA_NO_NODE): In such a case any node inside
> the domain's node mask is acceptable, but no other node. This changed
> behavior is (implicitly) being exposed through the memop hypercalls.
> 
> Note further that this change doesn't take care of moving the initrd
> image into memory matching Dom0's affinity when the initrd doesn't get
> copied (because of being part of the initial mapping) anyway.
> 
> Signed-off-by: Jan Beulich <jbeulich@xxxxxxxx>
>
Reviewed-by: Dario Faggioli <dario.faggioli@xxxxxxxxxx>

Just a couple of questions/comments.

> ---
> I'm uncertain whether range restricting the PXMs permitted for Dom0 is
> the right approach (matching what other NUMA code did until recently),
> or whether we would instead want to simply limit the number of PXMs we
> can handler there (i.e. using a static array instead of a static
> bitmap).
> 
FWIW, I think the approach taken in the patch is ok.

> --- a/docs/misc/xen-command-line.markdown
> +++ b/docs/misc/xen-command-line.markdown
> @@ -540,6 +540,15 @@ any dom0 autoballooning feature present 
>  _xl.conf(5)_ man page or [Xen Best
>  
> Practices](http://wiki.xen.org/wiki/Xen_Best_Practices#Xen_dom0_dedicated_memory_and_preventing_dom0_memory_ballooning).
>  
> +### dom0\_nodes
> +
> +> `= <integer>[,...]`
> +
> +Specify the NUMA nodes to place Dom0 on. Defaults for vCPU-s created
> +and memory assigned to Dom0 will be adjusted to match the node
> +restrictions set up here. Note that the values to be specified here are
> +ACPI PXM ones, not Xen internal node numbers.
> +
Why use PXM ids? It might be me being much more used to work with NUMA
node ids, but wouldn't the other way round be more consistent (almost
everything the user interacts with after boot speak node ids) and easier
for the user to figure things out (e.g., with tools like numactl on
baremetal)?

> --- a/xen/arch/x86/domain_build.c
> +++ b/xen/arch/x86/domain_build.c

> +static struct vcpu *__init setup_vcpu(struct domain *d, unsigned int vcpu_id,
> +                                      unsigned int cpu)
> +{
> +    struct vcpu *v = alloc_vcpu(d, vcpu_id, cpu);
> +
> +    if ( v )
> +    {
> +        if ( !d->is_pinned )
> +            cpumask_copy(v->cpu_hard_affinity, &dom0_cpus);
> +        cpumask_copy(v->cpu_soft_affinity, &dom0_cpus);
> +    }
> +
About this, for DomUs, now that we have soft affinity available, what we
do is set only soft affinity to match the NUMA placement. I think I see
and agree why we want to be 'more strict' in Dom0, but I felt like it
was worth to point out the difference in behaviour (should it be
documented somewhere?).

Regards,
Dario

BTW, mostly out of curiosity, I've had a few strange issues/conflicts in
applying this on top of staging, in order to test it... Was it me doing
something very stupid, or was this based on something different?

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel