Re: [PATCH 2/2] x86/cpuid: support LFENCE always serializing CPUID bit

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 14/04/2021 14:05, Andrew Cooper wrote:
> On 14/04/2021 13:57, Jan Beulich wrote:
>> On 14.04.2021 13:04, Roger Pau Monne wrote:
>>> @@ -264,6 +265,38 @@ struct cpuid_policy
>>>              };
>>>              uint32_t nc:8, :4, apic_id_size:4, :16;
>>>              uint32_t /* d */:32;
>>> +
>>> +            uint64_t :64, :64; /* Leaf 0x80000009. */
>>> +            uint64_t :64, :64; /* Leaf 0x8000000a - SVM rev and features. 
>>> */
>>> +            uint64_t :64, :64; /* Leaf 0x8000000b. */
>>> +            uint64_t :64, :64; /* Leaf 0x8000000c. */
>>> +            uint64_t :64, :64; /* Leaf 0x8000000d. */
>>> +            uint64_t :64, :64; /* Leaf 0x8000000e. */
>>> +            uint64_t :64, :64; /* Leaf 0x8000000f. */
>>> +            uint64_t :64, :64; /* Leaf 0x80000010. */
>>> +            uint64_t :64, :64; /* Leaf 0x80000011. */
>>> +            uint64_t :64, :64; /* Leaf 0x80000012. */
>>> +            uint64_t :64, :64; /* Leaf 0x80000013. */
>>> +            uint64_t :64, :64; /* Leaf 0x80000014. */
>>> +            uint64_t :64, :64; /* Leaf 0x80000015. */
>>> +            uint64_t :64, :64; /* Leaf 0x80000016. */
>>> +            uint64_t :64, :64; /* Leaf 0x80000017. */
>>> +            uint64_t :64, :64; /* Leaf 0x80000018. */
>>> +            uint64_t :64, :64; /* Leaf 0x80000019 - TLB 1GB Identifiers. */
>>> +            uint64_t :64, :64; /* Leaf 0x8000001a - Performance related 
>>> info. */
>>> +            uint64_t :64, :64; /* Leaf 0x8000001b - IBS feature 
>>> information. */
>>> +            uint64_t :64, :64; /* Leaf 0x8000001c. */
>>> +            uint64_t :64, :64; /* Leaf 0x8000001d - Cache properties. */
>>> +            uint64_t :64, :64; /* Leaf 0x8000001e - Extd APIC/Core/Node 
>>> IDs. */
>>> +            uint64_t :64, :64; /* Leaf 0x8000001f - AMD Secure Encryption. 
>>> */
>>> +            uint64_t :64, :64; /* Leaf 0x80000020 - Platform QoS. */
>>> +
>>> +            /* Leaf 0x80000021 - Extended Feature 2 */
>>> +            union {
>>> +                uint32_t e21a;
>>> +                struct { DECL_BITFIELD(e21a); };
>>> +            };
>>> +            uint32_t /* b */:32, /* c */:32, /* d */:32;
>>>          };
>>>      } extd;
>> Due to the effect of this on what guests get to see, I think this
>> wants to take my "x86/CPUID: shrink max_{,sub}leaf fields according
>> to actual leaf contents" as a prereq, which in turn may better
>> remain on top of "x86/CPUID: adjust extended leaves out of range
>> clearing" (both are neighbors in that over 4 months old series,
>> fair parts of which could imo go in irrespective of the unsettled
>> dispute on xvmalloc() - unfortunately I had made that patch 2 of
>> the series, not expecting it to be blocked for so long, and then
>> presumably signaling to others that the rest of the series is also
>> blocked).
> There is no shrinking to be done in this case.  The bit is set across
> the board on AMD/Hygon hardware, even on older parts.
>
> What does need changing however is the logic to trim max_extd_leaf down
> to what hardware supports, so the bit is visible on Rome/older
> hardware.  I.e. after this change, all VMs should get 0x80000021 by
> default on AMD hardware.
>
> (A curious observation of Milan hardware is that it actually advertises
> 0x80000023 as max_extd_leaf, and has two leaves of zeros at the end. 
> I've got an open query about this.)

Something like this:

diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c
index 050cd5713e..d9eb2878c5 100644
--- a/xen/arch/x86/cpuid.c
+++ b/xen/arch/x86/cpuid.c
@@ -311,6 +311,7 @@ static void __init calculate_raw_policy(void)
 static void __init calculate_host_policy(void)
 {
     struct cpuid_policy *p = &host_cpuid_policy;
+    unsigned int max_extd_leaf;
 
     *p = raw_cpuid_policy;
 
@@ -318,7 +319,18 @@ static void __init calculate_host_policy(void)
         min_t(uint32_t, p->basic.max_leaf,   ARRAY_SIZE(p->basic.raw) - 1);
     p->feat.max_subleaf =
         min_t(uint32_t, p->feat.max_subleaf, ARRAY_SIZE(p->feat.raw) - 1);
-    p->extd.max_leaf = 0x80000000 | min_t(uint32_t, p->extd.max_leaf &
0xffff,
+
+    max_extd_leaf = p->extd.max_leaf;
+
+    /*
+     * For AMD/Hygon hardware before Zen3, we modify LFENCE to be dispatch
+     * serialsing.  Extend max_extd_leaf beyond what hardware supports, to
+     * include the feature leaf containing this information.
+     */
+    if ( cpu_has_lfence_dispatch )
+        max_extd_leaf = max(max_extd_leaf, 0x80000021);
+
+    p->extd.max_leaf = 0x80000000 | min_t(uint32_t, max_extd_leaf & 0xffff,
                                           ARRAY_SIZE(p->extd.raw) - 1);
 
     cpuid_featureset_to_policy(boot_cpu_data.x86_capability, p);



Only compile tested.  Needs checking on Zen2 or older.  The Raw policy
should have the real max_extd_leaf, while host and derived should have
0x80000021.

~Andrew