Re: [Xen-devel] PV-vNUMA issue: topology is misinterpreted by the guest

[Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx>]

On 07/22/2015 10:09 AM, Juergen Gross wrote:
> On 07/22/2015 03:58 PM, Boris Ostrovsky wrote:
> > On 07/22/2015 09:50 AM, Juergen Gross wrote:
> > > On 07/22/2015 03:36 PM, Dario Faggioli wrote:
> > > > On Tue, 2015-07-21 at 16:00 -0400, Boris Ostrovsky wrote:
> > > > > On 07/20/2015 10:43 AM, Boris Ostrovsky wrote:
> > > > > > On 07/20/2015 10:09 AM, Dario Faggioli wrote:
> > > > > > I'll need to see how LLC IDs are calculated, probably also from some
> > > > > > CPUID bits.
> > > > >
> > > > > No, can't do this: LLC is calculated from CPUID leaf 4 (on Intel) 
> > > > > which
> > > > > use indexes in ECX register and xl syntax doesn't allow you to 
> > > > > override
> > > > > CPUIDs for such leaves.
> > > > Right. Which leaves us with the question of what should we do and/or
> > > > recommend users to do?
> > > >
> > > > If there were a workaround that we could put in place, and document
> > > > somewhere, however tricky it was, I'd say to go for it, and call it
> > > > acceptable for now.
> > > >
> > > > But, if there isn't, should we disable PV vnuma, or warn the user that
> > > > he may see issues? Can we identify, in Xen or in toolstack, whether an
> > > > host topology will be problematic, and disable/warn in those cases 
> > > > too?
> > > > I'm not sure, honestly. Disabling looks too aggressive, but it's an
> > > > issue I wouldn't like an user to be facing, without at least being
> > > > informed of the possibility... so, perhaps a (set of) warning(s)?
> > > > Thoughts?
> > > I think we have 2 possible solutions:
> > >
> > > 1. Try to handle this all in the hypervisor via CPUID mangling.
> > >
> > > 2. Add PV-topology support to the guest and indicate this capability 
> > > via
> > >    elfnote; only enable PV-numa if this note is present.
> > >
> > > I'd prefer the second solution. If you are okay with this, I'd try 
> > > to do
> > > some patches for the pvops kernel.
Why do you think that kernel patches are preferable to CPUID management? 
This would be all in tools, I'd think. (Well, one problem that I can 
think of is that AMD sometimes pokes at MSRs and/or Northbridge's PCI 
registers to figure out nodeID --- that we may need to have to address 
in the hypervisor)
And those patches won't help HVM guests, will they? How would they be 
useful by user processes?

-boris

> > What if I configure a guest to follow HW topology? I.e. I pin VCPUs to
> > appropriate cores/threads? With elfnote I am stuck with disabled 
> > topology.
> Add an option to do exactly that: follow HW topology (pin vcpus,
> configure vnuma)?
>
> Add a force flag to the vnuma configuration to ignore the elfnote?
>
> > Besides, this is not necessarily a NUMA-only issue, it's a scheduling
> > one (inside the guest) as well.
> Sure. That's what Jan said regarding SUSE's xen-kernel. No toplogy info
> (or a trivial one) might be better than the wrong one...
>
> This patch for pvops should be written in any case. I'll do this, but it
> would be nice to know whether PV-numa should be considered or not.



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel