Re: [XEN PATCH] tools: add x2APIC entries in MADT based on APIC ID

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Mon, Mar 25, 2024 at 02:30:35PM +0000, Alejandro Vallejo wrote:
> Hi,
> 
> On 14/03/2024 13:50, Jan Beulich wrote:
> > On 13.03.2024 16:35, Matthew Barnes wrote:
> >> libacpi is a tool that is used by libxl (for PVH guests) and hvmloader
> >> (for HVM guests) to construct ACPI tables for guests.
> >>
> >> Currently, libacpi only uses APIC entries to enumerate processors for
> >> guests in the MADT.
> >>
> >> The APIC ID field in APIC entries is an octet big, which is fine for
> >> xAPIC IDs, but not so for sufficiently large x2APIC IDs.
> > 
> > Yet where would those come from? I can see that down the road we will
> > have such, but right now I don't think we do. Without saying so, this
> > change could be mistaken for a fix of an active bug.
> 
> It's worth adding some context here.
> 
> You're quite right in that it's not immediately needed now, but with the
> work done on improving the state of CPU topologies exposed to guests[1]
> the strict binding between APIC ID and vCPU ID breaks. It's not hard to
> imagine sparsity in the APIC ID space forcing the maximum one to peak
> beyond 254. The generator present in that series tries to be
> conservative and avoid it, but general topologies can theoretically
> waste copious amounts of APIC ID space (i.e: by reserving more width
> than strictly required to represent IDs of a certain level), and
> exposing the host topology sensibly becomes difficult if we're subject
> to limitations the host does not have.

Most guest OSes will refuse to bring up APs with APIC ID > 254, unless
there's support for interrupt remapping or Extended Destination ID
[0].  So while adding the entries in the MADT is fine, I wonder how we
can sensibly test this.

IMO, we should first add support for Extended Destination ID, and then
expose APs with APIC ID > 254.

Have you been able to test this with APIC IDs > 254, and can confirm
that an OS is capable of bringing those APs up with the ID in the
x2APIC MADT entry?

I think at a minimum you need some changes to the vlapic code in Xen,
so that when a vLAPIC gets assigned an ID > 254 it automatically
switches to x2APIC mode, as vlapic_init() unconditionally inits the
lapic in xAPIC mode.  Otherwise the INTI-SIPI-SIPI sequence won't find
the intended destination.

Thanks, Roger.

[0] https://gitlab.com/xen-project/xen/-/issues/173