Re: [Xen-devel] [PATCH v2 08/16] xen: derive NUMA node affinity from hard and soft CPU affinity

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

On ven, 2013-11-15 at 10:52 +0000, George Dunlap wrote:
> On 14/11/13 16:30, Dario Faggioli wrote:
>
> > I'm not sure I fully get what you mean... I cannot afford neglecting
> > online_affinity, independently from how cpumask and cpumask_soft look
> > like, because that's what's needed to account for cpupools. Anyway, I'll
> > think more about it and see if I can make it better.
> 
> So what you have here is (in pseudocode):
> 
>   online_affinity = hard_affinity & online;
>   cpumask |= online_affinity;
>   cpumask_soft |= soft_affinity;
> 
> if ( intersects(cpumask, cpumask_soft) )
>    cpumask &= cpumask_soft;
> 
> So at least four full bitwise operations, plus "intersects" which will 
> be the equivalent of a full bitwise operation if it turns out to be false.
> 
Well, that's not exactly what I have. Fact is, _every_ vcpu has a
soft_affinity and a hard_affinity. That's why I have the cpumask_or()-s
in the loop, to build a hard haffinity and a soft affinity mask of the
_domain_, by ||-ing all the hard (soft) affinity of all the vcpus.

From the pseudo code above (and even from the one below), it's not clear
to me whether you mean vcpu hard (and soft) affinity or domain ones.

In fact...

> How about something like the following:
> 
> cpumask_hard = hard_affinity & online;
> cpumask_soft = cpumask_hard & soft_affinity;
> 
> cpumask_p = is_empty(cpumask_soft) ? &cpumask_hard : &cpumask_soft;
> 
... From reading this last pseudo-statement I think you actually mean
the affinity masks for the whole domain, which is exactly what I also
want to consider... But I have to construct them somehow... That's,
again, form where the cpumask_or()-s come.

So, thanking this per-vcpu-ness into account, what I have is:

  for_each_vcpu(i)
    dom_cpumask_hard |= online & cpumask_hard(i)
    dom_cpumask_soft |= cpumask_soft(i)

  if ( dom_cpumask_hard & dom_cpumask_soft )
    dom_cpumask = dom_cpumask_hard & dom_cpumask_soft
  else
    <use dom_cpumask_hard>

Perhaps I can rename the variable with something like this 'dom_' prefix
in the real code too, to make things more clear.

However, it is indeed true that online is per domain, so yes, perhaps I
can ddo something like the following:

  for_each_vcpu(i)
    dom_cpumask_hard |= cpumask_hard(i)
    dom_cpumask_soft |= cpumask_soft(i)

  dom_cpumask_hard &= online
  dom_cpumask_soft &= dom_cpumask_hard

  dom_cpumask_p = is_empty(dom_cpumask_soft) ?
                      dom_cpumask_hard : dom_cpumask_soft;

I think it's both more efficient and more clear than what I have now,
and I give you that it's close enough to what you were suggesting... But
I don't think I can ditch the '|=' in the loop for building up the two
domain wide cpumasks.

How do you like it?

Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel