[Xen-devel] [PATCH v2] x86/boot: Disable IBRS in intr/nmi exit path at bootup stage

[Zhenzhong Duan <zhenzhong.duan@xxxxxxxxxx>]

After reset, IBRS is disabled by processor, but a coming intr/nmi leave IBRS
enabled after their exit. It's not necessory for bootup code to run in low
performance with IBRS enabled.

On ORACLE X6-2(500GB/88 cpus, dom0 11GB/20 vcpus), we observed an 200s+ delay
in construct_dom0.

By initializing use_shadow_spec_ctrl with the result of (system_state <
SYS_STATE_active), IBRS is disabled in intr/nmi exit path at bootup stage.
Then delay in construct_dom0 is ~50s.

When hot-onlining a CPU, we initialize IBRS early and set use_shadow_spec_ctrl
to false to avoid Branch Target Injection from sibling threads.

v2: Use (system_state < SYS_STATE_active) to initialize use_shadow_spec_ctrl
instead of literal 1 per Jan.

Signed-off-by: Zhenzhong Duan <zhenzhong.duan@xxxxxxxxxx>
---
 xen/include/asm-x86/spec_ctrl.h |   16 +++++++++++++++-
 1 files changed, 15 insertions(+), 1 deletions(-)

diff --git a/xen/include/asm-x86/spec_ctrl.h b/xen/include/asm-x86/spec_ctrl.h
index 5ab4ff3..1672317 100644
--- a/xen/include/asm-x86/spec_ctrl.h
+++ b/xen/include/asm-x86/spec_ctrl.h
@@ -32,8 +32,22 @@ extern uint8_t default_bti_ist_info;
 static inline void init_shadow_spec_ctrl_state(void)
 {
     struct cpu_info *info = get_cpu_info();
+    uint32_t val = SPEC_CTRL_IBRS;
+
+    /* Initialize IA32_SPEC_CTRL MSR for hotplugging cpu early */
+    if ( system_state >= SYS_STATE_active )
+        asm volatile (ALTERNATIVE(ASM_NOP3, "wrmsr", X86_FEATURE_XEN_IBRS_SET)
+                      :: "a" (val), "c" (MSR_SPEC_CTRL), "d" (0) : "memory");
 
-    info->shadow_spec_ctrl = info->use_shadow_spec_ctrl = 0;
+    info->shadow_spec_ctrl = 0;
+    /*
+     * We want to make sure we clear IBRS in interrupt exit path
+     * (DO_SPEC_CTRL_EXIT_TO_XEN) while dom0 is still booting to
+     * avoid unnecessary performance impact. As soon as dom0 has
+     * booted use_shadow_spec_ctrl will be cleared, for example,
+     * in idle routine.
+     */
+    info->use_shadow_spec_ctrl = system_state < SYS_STATE_active;
     info->bti_ist_info = default_bti_ist_info;
 }
 
-- 
1.7.3

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel