Re: [Xen-users] "Booted on L1TF-vulnerable hardware with SMT/Hyperthreading enabled" .. or not?

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 02/09/18 23:27, Hans van Kranenburg wrote:
> On 09/03/2018 12:14 AM, Andrew Cooper wrote:
>> On 02/09/18 21:22, Hans van Kranenburg wrote:
>>> On 08/31/2018 02:12 AM, Hans van Kranenburg wrote:
>>>> On 08/31/2018 02:09 AM, Hans van Kranenburg wrote:
>>>>> I have some HP ProLiant DL360 G7 (P68) servers here, and I just put this
>>>>> BIOS update on them:
>>>>>
>>>>> https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_23267b7aabb6489a8332d06919#tab3
>>>>>
>>>>> Version: 2018.05.21(2 Jul 2018)
>>>>>
>>>>> The release notes contain a whole story about "This revision of the
>>>>> System ROM includes the latest revision of the Intel microcode which, in
>>>>> combination with operating system and hypervisor updates, provides
>>>>> mitigation for the L1 Terminal Fault – OS/SMM (CVE-2018-3620) and L1
>>>>> Terminal Fault – VMM (CVE-2018-3646) security vulnerabilities."
>>>>>
>>>>> In the BIOS cpu settings, hyperthreading is disabled. This server
>>>>> contains 2x 6-core Intel(R) Xeon(R) CPU X5675 @ 3.07GHz
>>>>>
>>>>> From xen info: (4.11, built from stable-4.11 commit 733450b39b)
>>>>>
>>>>> -# xen info
>>>>> host                   : rho
>>>>> release                : 4.17.0-0.bpo.3-amd64
>>>>> version                : #1 SMP Debian 4.17.17-1~bpo9+1 (2018-08-27)
>>>>> machine                : x86_64
>>>>> nr_cpus                : 12
>>>>> max_cpu_id             : 31
>>>>> nr_nodes               : 2
>>>>> cores_per_socket       : 6
>>>>> threads_per_core       : 1
>>> From server startup serial console:
>>>
>>> ---- >8 ----
>>>                             108 GB Installed
>>>
>>>
>>>
>>> ProLiant System BIOS - P68 (05/21/2018)
>>>
>>> Copyright 1982, 2018 Hewlett-Packard Development Company, L.P.
>>>
>>>
>>>
>>>
>>> 2 Processor(s) detected, 12 total cores enabled, Hyperthreading is disabled
>>> Proc 1: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz
>>> Proc 2: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz
>>> QPI Speed: 6.4 GT/s
>>> HP Power Profile Mode: Custom
>>> Power Regulator Mode: Static High Performance
>>>
>>> Advanced Memory Protection Mode: Advanced ECC Support
>>> Redundant ROM Detected - This system contains a valid backup system ROM.
>>> Inlet Ambient Temperature: 20C/68F
>>>
>>> ---- >8 ----
>>>
>>> "Hyperthreading is disabled"
>>>
>>>>> Still, xl dmesg shows me:
>>>>>
>>>>> (XEN) ***************************************************
>>>>> (XEN) Booted on L1TF-vulnerable hardware with SMT/Hyperthreading
>>>>> (XEN) enabled.  Please assess your configuration and choose an
>>>>> (XEN) explicit 'smt=<bool>' setting.  See XSA-273.
>>>>> (XEN) ***************************************************
>>>>>
>>>>> What's wrong here?
>>>> Additionally:
>>> So, interestingly, this information (from dom0) lists ht again in flags:
>>>
>>>> -# cat /proc/cpuinfo
>>>> processor  : 0
>>>> vendor_id  : GenuineIntel
>>>> cpu family : 6
>>>> model              : 44
>>>> model name : Intel(R) Xeon(R) CPU           X5675  @ 3.07GHz
>>>> stepping   : 2
>>>> microcode  : 0x1f
>>>> cpu MHz            : 3066.788
>>>> cache size : 12288 KB
>>>> physical id        : 0
>>>> siblings   : 4
>>>> core id            : 0
>>>> cpu cores  : 1
>>>> apicid             : 0
>>>> initial apicid     : 0
>>>> fpu                : yes
>>>> fpu_exception      : yes
>>>> cpuid level        : 11
>>>> wp         : yes
>>>> flags              : fpu de tsc msr pae mce cx8 apic sep mca cmov pat 
>>>> clflush acpi
>>>> mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl
>>>> nonstop_tsc cpuid pni pclmulqdq monitor est ssse3 cx16 sse4_1 sse4_2
>>>> popcnt aes hypervisor lahf_lm ssbd ibrs ibpb stibp
>>>> bugs               : null_seg cpu_meltdown spectre_v1 spectre_v2 
>>>> spec_store_bypass l1tf
>>>> bogomips   : 6133.57
>>>> clflush size       : 64
>>>> cache_alignment    : 64
>>>> address sizes      : 40 bits physical, 48 bits virtual
>>>> power management:
>>>>
>>>> (identical for all cpus listed)
>>> Another view on the thing, from dmidecode:
>>>
>>> Handle 0x0400, DMI type 4, 42 bytes
>>> Processor Information
>>>     Socket Designation: Proc 1
>>>     Type: Central Processor
>>>     Family: Xeon
>>>     Manufacturer: Intel
>>>     ID: C2 06 02 00 FF FB EB BF
>>>     Signature: Type 0, Family 6, Model 44, Stepping 2
>>>     Flags:
>>>             FPU (Floating-point unit on-chip)
>>>             VME (Virtual mode extension)
>>>             DE (Debugging extension)
>>>             PSE (Page size extension)
>>>             TSC (Time stamp counter)
>>>             MSR (Model specific registers)
>>>             PAE (Physical address extension)
>>>             MCE (Machine check exception)
>>>             CX8 (CMPXCHG8 instruction supported)
>>>             APIC (On-chip APIC hardware supported)
>>>             SEP (Fast system call)
>>>             MTRR (Memory type range registers)
>>>             PGE (Page global enable)
>>>             MCA (Machine check architecture)
>>>             CMOV (Conditional move instruction supported)
>>>             PAT (Page attribute table)
>>>             PSE-36 (36-bit page size extension)
>>>             CLFSH (CLFLUSH instruction supported)
>>>             DS (Debug store)
>>>             ACPI (ACPI supported)
>>>             MMX (MMX technology supported)
>>>             FXSR (FXSAVE and FXSTOR instructions supported)
>>>             SSE (Streaming SIMD extensions)
>>>             SSE2 (Streaming SIMD extensions 2)
>>>             SS (Self-snoop)
>>>             HTT (Multi-threading)
>>>             TM (Thermal monitor supported)
>>>             PBE (Pending break enabled)
>>>     Version: Intel(R) Xeon(R) CPU X5675 @ 3.07GHz
>>>     Voltage: 1.4 V
>>>     External Clock: 133 MHz
>>>     Max Speed: 4800 MHz
>>>     Current Speed: 3067 MHz
>>>     Status: Populated, Enabled
>>>     Upgrade: Socket LGA1366
>>>     L1 Cache Handle: 0x0710
>>>     L2 Cache Handle: 0x0720
>>>     L3 Cache Handle: 0x0730
>>>     Serial Number: Not Specified
>>>     Asset Tag: Not Specified
>>>     Part Number: Not Specified
>>>     Core Count: 6
>>>     Core Enabled: 6
>>>     Thread Count: 12
>>>     Characteristics:
>>>             64-bit capable
>>>
>>> This also lists hyperthreading and 6 cores with 12 threads per physical
>>> cpu...
>>>
>>> The code which decides about printing the message or not does:
>>>
>>> if ( opt_smt == -1 && cpu_has_bug_l1tf && !pv_shim &&
>>>      boot_cpu_data.x86_num_siblings > 1 )
>>>     warning_add(
>>>         "Booted on L1TF-vulnerable hardware with SMT/Hyperthreading\n"
>>>         "enabled.  Please assess your configuration and choose an\n"
>>>         "explicit 'smt=<bool>' setting.  See XSA-273.\n");
>>>
>>> (from commit 2a47c75509, author in Cc)
>>>
>>> Apparently in here, boot_cpu_data.x86_num_siblings > 1 still matches
>>> when hyperthreading is actually disabled in bios configuration?
>>>
>>> Is there a better way to detect configuration (enabled/disabled) vs.
>>> just the fact that the hardware has the feature and could enable it?
>>> Something like the same source as where xl info gets the
>>> threads_per_core from, which does show up as 1?
>> The HT flag in CPUID is of no use in this case.  It is actually
>> HyperThreadingTechnology which is a bit signalling that other topology
>> related information in the CPUID leaves has changed from the legacy
>> enumeration mechanisms.  Most hardware in the last 10 years sets the HTT
>> bit, even on non-HT capable hardware.
>>
>> Do you have the full Xen boot log (`xl dmesg` once dom0 has booted) ?
>>
>> Xen's boot_cpu_data.x86_num_siblings ought to be correct, but I'd agree
>> in this case it appears to be in direct contradiction to what the
>> firmware claims.
> (XEN) parameter "placeholder" unknown!
> (XEN) Xen version 4.11.1-pre (Debian
> 4.11.1~pre+1.733450b39b-1~exp1~mxbp9+1) (hans@xxxxxxxxxxx) (gcc (Debian
> 6.3.0-18+deb9u1) 6.3.0 20170516) debug=n  Wed Aug 22 15:26:44 UTC 2018
> (XEN) Bootloader: GRUB 2.02~beta3-5
> (XEN) Command line: placeholder dom0_max_vcpus=4 dom0_mem=4G,max:4G
> com2=115200,8n1 console=com2,vga noreboot xpti=no-dom0
> (XEN) Xen image load base address: 0
> (XEN) Video information:
> (XEN)  VGA is text mode 80x25, font 8x16
> (XEN)  VBE/DDC methods: none; EDID transfer time: 2 seconds
> (XEN)  EDID info not retrieved because no DDC retrieval method detected
> (XEN) Disc information:
> (XEN)  Found 1 MBR signatures
> (XEN)  Found 1 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN)  0000000000000000 - 000000000009f400 (usable)
> (XEN)  000000000009f400 - 00000000000a0000 (reserved)
> (XEN)  00000000000f0000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 00000000d762f000 (usable)
> (XEN)  00000000d762f000 - 00000000d763c000 (ACPI data)
> (XEN)  00000000d763c000 - 00000000d763d000 (usable)
> (XEN)  00000000d763d000 - 00000000dc000000 (reserved)
> (XEN)  00000000fec00000 - 00000000fee10000 (reserved)
> (XEN)  00000000ff800000 - 0000000100000000 (reserved)
> (XEN)  0000000100000000 - 0000001b27fff000 (usable)
> (XEN) New Xen image base address: 0xd7000000
> (XEN) ACPI: RSDP 000F4F00, 0024 (r2 HP    )
> (XEN) ACPI: XSDT D7630140, 00B4 (r1 HP     ProLiant        2   �     162E)
> (XEN) ACPI: FACP D7630240, 00F4 (r3 HP     ProLiant        2   �     162E)
> (XEN) ACPI: DSDT D7630340, 20BD (r1 HP         DSDT        1 INTL 20030228)
> (XEN) ACPI: FACS D762F100, 0040
> (XEN) ACPI: SPCR D762F140, 0050 (r1 HP     SPCRRBSU        1   �     162E)
> (XEN) ACPI: MCFG D762F1C0, 003C (r1 HP     ProLiant        1             0)
> (XEN) ACPI: HPET D762F200, 0038 (r1 HP     ProLiant        2   �     162E)
> (XEN) ACPI: FFFF D762F240, 0064 (r2 HP     ProLiant        2   �     162E)
> (XEN) ACPI: SPMI D762F2C0, 0040 (r5 HP     ProLiant        1   �     162E)
> (XEN) ACPI: ERST D762F300, 01D0 (r1 HP     ProLiant        1   �     162E)
> (XEN) ACPI: APIC D762F500, 015E (r1 HP     ProLiant        2             0)
> (XEN) ACPI: SRAT D762F680, 0570 (r1 HP     Proliant        1   �     162E)
> (XEN) ACPI: FFFF D762FC00, 0176 (r1 HP     ProLiant        1   �     162E)
> (XEN) ACPI: BERT D762FD80, 0030 (r1 HP     ProLiant        1   �     162E)
> (XEN) ACPI: HEST D762FDC0, 00BC (r1 HP     ProLiant        1   �     162E)
> (XEN) ACPI: DMAR D762FE80, 0146 (r1 HP     ProLiant        1   �     162E)
> (XEN) ACPI: SSDT D7632400, 0125 (r3     HP  CRSPCI0        2   HP        1)
> (XEN) ACPI: SSDT D7632540, 01CF (r3     HP  riser1a        2 INTL 20061109)
> (XEN) ACPI: SSDT D7632740, 03BB (r1     HP      pcc        1 INTL 20090625)
> (XEN) ACPI: SSDT D7632B00, 0377 (r1     HP     pmab        1 INTL 20090625)
> (XEN) ACPI: SSDT D7632E80, 2094 (r1  INTEL PPM RCM         1 INTL 20061109)
> (XEN) System RAM: 110581MB (113235768kB)
> (XEN) Domain heap initialised DMA width 32 bits
> (XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
> (XEN) IOAPIC[1]: apic_id 0, version 32, address 0xfec80000, GSI 24-47
> (XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs
> (XEN) Failed to get Error Log Address Range.
> (XEN) Speculative mitigation facilities:
> (XEN)   Hardware features: IBRS/IBPB STIBP L1D_FLUSH SSBD
> (XEN)   Compiled-in support: INDIRECT_THUNK SHADOW_PAGING
> (XEN)   Xen settings: BTI-Thunk RETPOLINE, SPEC_CTRL: IBRS- SSBD-,
> Other: IBPB L1D_FLUSH
> (XEN)   L1TF: believed vulnerable, maxphysaddr L1D 44, CPUID 40, Safe
> address 10000000000
> (XEN)   Support for VMs: PV: MSR_SPEC_CTRL RSB EAGER_FPU, HVM:
> MSR_SPEC_CTRL RSB EAGER_FPU
> (XEN)   XPTI (64-bit PV only): Dom0 disabled, DomU disabled
> (XEN)   PV L1TF shadowing: Dom0 disabled, DomU enabled
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Detected 3066.791 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) [VT-D]Disabling IOMMU due to Intel 5500/5520/X58 Chipset errata
> #47, #53
> (XEN) I/O virtualisation disabled
> (XEN) Enabled directed EOI with ioapic_ack_old on!
> (XEN) ENABLING IO-APIC IRQs
> (XEN)  -> Using old ACK method
> (XEN) Allocated console ring of 32 KiB.
> (XEN) VMX: Supported advanced features:
> (XEN)  - APIC MMIO access virtualisation
> (XEN)  - APIC TPR shadow
> (XEN)  - Extended Page Tables (EPT)
> (XEN)  - Virtual-Processor Identifiers (VPID)
> (XEN)  - Virtual NMI
> (XEN)  - MSR direct-access bitmap
> (XEN)  - Unrestricted Guest
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: VMX enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) detected
> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> (XEN) Brought up 12 CPUs
> (XEN) Dom0 has maximum 816 PIRQs
> (XEN)  Xen  kernel: 64-bit, lsb, compat32
> (XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x266a000
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN)  Dom0 alloc.:   0000001adc000000->0000001ae0000000 (1025804 pages
> to be allocated)
> (XEN)  Init. ramdisk: 0000001b2650c000->0000001b27dffa74
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN)  Loaded kernel: ffffffff81000000->ffffffff8266a000
> (XEN)  Init. ramdisk: 0000000000000000->0000000000000000
> (XEN)  Phys-Mach map: 0000008000000000->0000008000800000
> (XEN)  Start info:    ffffffff8266a000->ffffffff8266a4b4
> (XEN)  Xenstore ring: 0000000000000000->0000000000000000
> (XEN)  Console ring:  0000000000000000->0000000000000000
> (XEN)  Page tables:   ffffffff8266b000->ffffffff82682000
> (XEN)  Boot stack:    ffffffff82682000->ffffffff82683000
> (XEN)  TOTAL:         ffffffff80000000->ffffffff82800000
> (XEN)  ENTRY ADDRESS: ffffffff82433180
> (XEN) Dom0 has maximum 4 VCPUs
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Scrubbing Free RAM on 2 nodes using 12 CPUs
> (XEN)
> .........................................................................done.
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> (XEN) ***************************************************
> (XEN) Booted on L1TF-vulnerable hardware with SMT/Hyperthreading
> (XEN) enabled.  Please assess your configuration and choose an
> (XEN) explicit 'smt=<bool>' setting.  See XSA-273.
> (XEN) ***************************************************
> (XEN) 3... 2... 1...
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
> input to Xen)
> (XEN) Freed 476kB init memory
>
> I can reboot with another loglevel if needed.

Can you rerun with "cpuinfo" on the command line, and/or a debug build,
so we can see Xen's interpretation of the MADT ACPI table.

Additionally, if you can boot with "opt_dom0_vcpu_pin" and collect
`cpuid -r` which should pick up the raw processor APIC IDs.

Thanks,

~Andrew

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-users