Re: [Xen-devel] Xen HVM regression on certain Intel CPUs

[Stefan Bader <stefan.bader@xxxxxxxxxxxxx>]

On 27.03.2013 17:09, H. Peter Anvin wrote:
> On 03/27/2013 09:04 AM, Konrad Rzeszutek Wilk wrote:
>>>>
>>>> From the Xen debugging console it was possible to gather a bit more data 
>>>> which
>>>> pointed to a failure very close to setting CR4 in startup_32. On this 
>>>> particular
>>>> hardware the saved CR4 (about to be set) was 0x1407f0.
>>>>
>>>> This would set two flags that somehow feel dangerous: PGE (page global 
>>>> enable)
>>>> and SMEP (supervisor mode execution protection). SMEP turns out to be the 
>>>> main
>>>> offender and the following change allows the APs to start:
>>>>
>>>> --- a/arch/x86/realmode/rm/trampoline_64.S
>>>> +++ b/arch/x86/realmode/rm/trampoline_64.S
>>>> @@ -93,7 +93,9 @@ ENTRY(startup_32)
>>>>         movl    %edx, %fs
>>>>         movl    %edx, %gs
>>>>
>>>> -       movl    pa_tr_cr4, %eax
>>>> +       movl    $X86_CR4_SMEP, %eax
>>>> +       notl    %eax
>>>> +       andl    pa_tr_cr4, %eax
>>>>         movl    %eax, %cr4              # Enable PAE mode
>>>>
>>>>         # Setup trampoline 4 level pagetables
>>>>
>>>> Now I am not completely convinced that this is really the way to go. 
>>>> Likely the
>>>> Xen hypervisor should not start up the guest with CR4 on the BP containing 
>>>> those
>>>> flags. But maybe it still makes sense to mask some dangerous ones off in 
>>>> the
>>>> realmode code (btw, it seemed that masking the assignments in arch_setup or
>>>> setup_realmode did not work).
>>>>
>>>> And finally I am wondering why the SMEP flag in CR4 is set anyway. My
>>>> understanding would be that this should only be done if cpuid[7].ebx has 
>>>> bit7
>>>> set. And this does not seem to be the case at least on the one box I was 
>>>> doing
>>>> the bisection on.
>>>
>>> Seems that I was relying on the wrong source of information when checking 
>>> SMEP
>>> support. The cpuid command seems at fail. But /proc/cpuinfo reports it. So 
>>> that
>>> at least explains where that comes from... sorry for that.
>>
>> OK, so if you boot Xen with smep=1 (which disables SMEP, kind of 
>> counterintuive flag)
>> that would work fine?
>>
>> CC-ing the Intel folks who added this in.
>>
> 
> If it is present in /proc/cpuinfo and not in cpuid it means the kernel
> thinks it has SMEP but the CPU doesn't... an obvious case of fail.
> However, *where the hell* does the bit come from in the first place?

I did not yet have time to track down all sources but I thought that
/proc/cpuinfo is in some way assembled from whatever cpuid info the kernel has.
I am more suspicious of the cpuid command I was using. Let me check for x86info.

> 
> That is what we need to track down.
> 
> When you say Xen HVM, am I correct in assuming that neither CPUID nor
> CR4 operations in the main kernel are run through paravirt_ops?

Not paravirt ops likely but the hypervisor traps access. At least cpuid from
within a hvm guest I expect to be filtered. So when checking things I went to
bare-metal.

Will fetch more info and get back.

-Stefan
> 
>       -hpa
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel
>

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel