Re: [Xen-devel] [patch] x86: Add a delay between INIT & SIPIs for tboot AP bring-up in X2APIC case

["Wei, Gang" <gang.wei@xxxxxxxxx>]

Jan Beulich wrote on 2011-12-23:
>>>> On 23.12.11 at 04:14, "Wei, Gang" <gang.wei@xxxxxxxxx> wrote:
>> Without this delay, Xen could not bring APs up while working with
>> TXT/tboot, because tboot need some time in APs to handle INIT before
>> becoming ready for receiving SIPIs.(this delay was removed as part
>> of c/s 23724 by Tim Deegan)
>> 
>> Signed-off-by: Gang Wei <gang.wei@xxxxxxxxx>
>> Acked-by: Keir Fraser <keir@xxxxxxx>
>> Acked-by: Tim Deegan <tim.deegan@xxxxxxxxxx>
>> 
>> diff -r d1aefee43af1 xen/arch/x86/smpboot.c
>> --- a/xen/arch/x86/smpboot.c Wed Dec 21 18:51:31 2011 +0800
>> +++ b/xen/arch/x86/smpboot.c Fri Dec 23 11:01:10 2011 +0800
>> @@ -42,6 +42,7 @@
>>  #include <asm/msr.h> #include <asm/mtrr.h> #include <asm/time.h>
>>  +#include <asm/tboot.h> #include <mach_apic.h> #include
>>  <mach_wakecpu.h> #include <smpboot_hooks.h>
>> @@ -463,6 +464,18 @@
>>              send_status = apic_read(APIC_ICR) & APIC_ICR_BUSY;
>>          } while ( send_status && (timeout++ < 1000) );
>>      }
>> +    else if ( tboot_in_measured_env() )
>> +    {
>> +        /*
>> +         * With tboot AP is actually spinning in a mini-guest before
>> +         * receiving INIT. Upon receiving INIT ipi, AP need time to
>> + VMExit,
>> 
>> +         * update VMCS to tracking SIPIs and VMResume.
>> +         *
>> +         * While AP is in root mode handling the INIT the CPU will drop
>> +         * any SIPIs
>> +         */
>> +        udelay(10);
> 
> So you jumped from 100ms to 10us - how was that value established?
> Or in other words, how certain is it that this (or any other) timeout
> is sufficient for all current and future systems implementing tboot?

First way, code analysis. Tboot VMExitHandler only judge the condition, enable 
trapping SIPI, then VMResume. 10us means more than 10,000 cycles in Intel 
processors supporting TXT, it should be enough for this simple code path.

Second way, I tested it. The minimum value I tested is udelay(1). So I think 
udelay(10) should give enough margin.

Further, I am working on changing the SMP bring-up sequence for tboot path from 
INIT-SIPI-SIPI to MWAIT-memwrite style. It means tboot APs will wait in MWAIT 
loops and Xen BSP will try to write the monitored memory to bring APs out of 
MWAIT loops.

Jimmy



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel