Re: [Xen-devel] HVM CPU enumeration, mapping to VCPU ID (Was: Re: [Xen-users] FreeBSD PVHVM call for testing)

[Matt Wilson <msw@xxxxxxxxxx>]

On Fri, May 31, 2013 at 09:21:50AM +0100, Ian Campbell wrote:
> On Thu, 2013-05-30 at 10:16 -0700, Matt Wilson wrote:
> > 
> > On bare metal x86 Linux, the kernel enumerates CPUs based on an order
> > defined by the BIOS.
> >  Typically this means that all the cores are
> > enumerated first, followed by logical processors (HT/SMT). For Linux,
> > maxcpus=N/2 should disable HT on systems that enumerate processors in
> > the recommended order. Some history:
> >   https://bugzilla.kernel.org/show_bug.cgi?id=2317
> 
> How the guest chooses to enumerate the CPUs is not terribly relevant so
> long as the Xen specific code for that OS knows how to invert that
> mapping to get at the underlying ABI which determines Xen's VCPUID for a
> CPU.

Indeed.

> I think I was wrong to focus on the guest enumeration scheme before,
> what actually matters is where in our ABI we expose the VCPUID, which
> isn't at all clear to me.

Agreed.

> > The virtual BIOS provides both ACPI tables and a legacy MP-table that
> > gives the LAPIC id mapping. The guest could infer the Xen vCPU ID from
> > a processor's position in these tables.
> 
> Do we consider the ordering given in any of those tables to be an HVM
> guest ABI? What about the lapic_id == 2*vcpuid -- is that multiplication
> factor part of the ABI (i.e. is the guest expected to pass lapic_id/2 to
> vcpuop)?

I strongly prefer the order in the BIOS tables, *not* the
lapic_id = 2*vcpuid formula. Once I've done some libxl work, I'll be
proposing a patch that makes the LAPIC / x2APIC IDs configurable,
and that will break this assumption.

> >  Or we could add a VCPUOP that an enlightened guest could use to get
> > the information more directly.
> 
> I'm hoping that there is some existing interface which I simply don't
> know about, but yes this could be the answer if such a thing doesn't
> exist.

I don't know of one that provide the information explicitly. It might
be easiest to provide this as a hypervisor CPUID leaf so it can be
used in early boot.

> > One question: why does a hypercall take a parameter that only has one
> > valid value? That value can be determined by looking at the current
> > running vCPU.
> 
> The generic prototype is:
>         vcpu_op(int cmd, int vcpuid, void *extra_args)
> Some cmds can act on any vcpuid and others can only act on the current
> vcpu. In an ideal world we would have had VCPUID_SELF or something but
> its a bit late for that.

Yea, that makes sense.

> > The *2 is just for assigning the LAPIC ID, and I'm pretty sure that
> > Linux is assigning processor IDs sequentially at ACPI parse time.
> 
> That probably doesn't matter, what matters is the Xen specific parts of
> the kernel's ability to reverse that assignment to get at the underlying
> APIC ID, assuming that is actually an ABI from which we can infer the
> VCPU ID...

Indeed. This seems to be loosely defined so far, and easy to get wrong
as happened with this FreeBSD work.

Konrad, Keir - any thoughts here?

--msw

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel