[Xen-devel] Xen 4.9 vmentry failure

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

Hello,

I've found a second instance of this vmentry failure, this time from a
Windows 7 32bit VM.

(XEN) d18v1 vmentry failure (reason 0x80000021): Invalid guest state (0)
(XEN) ************* VMCS Area **************
(XEN) *** Guest State ***
(XEN) CR0: actual=0x000000008001003b, shadow=0x000000008001003b,
gh_mask=ffffffffffffffff
(XEN) CR4: actual=0x00000000000426f9, shadow=0x00000000000406f9,
gh_mask=ffffffffffffffff
(XEN) CR3 = 0x0000000000185000
(XEN) PDPTE0 = 0x0000000000186001  PDPTE1 = 0x0000000000187001
(XEN) PDPTE2 = 0x0000000000188001  PDPTE3 = 0x0000000000189001
(XEN) RSP = 0x00000000807cfba0 (0x00000000807cfba0)  RIP =
0x0000000082a411dc (0x0000000082a411dc)
(XEN) RFLAGS=0x00004002 (0x00004002)  DR7 = 0x0000000000000400
(XEN) Sysenter RSP=00000000807e8000 CS:RIP=0008:000000008265a300
(XEN)        sel  attr  limit   base
(XEN)   CS: 0008 0c09b ffffffff 0000000000000000
(XEN)   DS: 0023 0c0f3 ffffffff 0000000000000000
(XEN)   SS: 0023 0c0f3 ffffffff 0000000000000000
(XEN)   ES: 0023 0c0f3 ffffffff 0000000000000000
(XEN)   FS: 003b 040f3 00004000 000000007ffde000
(XEN)   GS: 0000 1c000 ffffffff 0000000000000000
(XEN) GDTR:            000003ff 00000000807cfc20
(XEN) LDTR: 0000 10000 00000000 0000000000000000
(XEN) IDTR:            000007ff 00000000807d0020
(XEN)   TR: 0058 0008b 00000068 00000000807ccb30
(XEN) EFER = 0x0000000000000000  PAT = 0x0007010600070106
(XEN) PreemptionTimer = 0x00000000  SM Base = 0x00000000
(XEN) DebugCtl = 0x0000000000000000  DebugExceptions = 0x0000000000000000
(XEN) Interruptibility = 00000008  ActivityState = 00000000
(XEN) *** Host State ***
(XEN) RIP = 0xffff82d08031ef30 (vmx_asm_vmexit_handler)  RSP =
0xffff8300cf28ff90
(XEN) CS=e008 SS=0000 DS=0000 ES=0000 FS=0000 GS=0000 TR=e040
(XEN) FSBase=0000000000000000 GSBase=0000000000000000
TRBase=ffff82d0805e5580
(XEN) GDTBase=ffff82d080465000 IDTBase=ffff82d0805deac0
(XEN) CR0=000000008005003b CR3=00000002c94bc000 CR4=00000000000426e0
(XEN) Sysenter RSP=ffff8300cf28ffc0 CS:RIP=e008:ffff82d080368370
(XEN) EFER = 0x0000000000000000  PAT = 0x0000050100070406
(XEN) *** Control State ***
(XEN) PinBased=0000003f CPUBased=b6a065fa SecondaryExec=000000eb
(XEN) EntryControls=000051ff ExitControls=000fefff
(XEN) ExceptionBitmap=00060002 PFECmask=00000000 PFECmatch=00000000
(XEN) VMEntry: intr_info=80000b0a errcode=00000010 ilen=00000000
(XEN) VMExit: intr_info=00000000 errcode=00000000 ilen=00000003
(XEN)         reason=80000021 qualification=0000000000000000
(XEN) IDTVectoring: info=80000202 errcode=00000000
(XEN) TSC Offset = 0xfffff8fad6357a07  TSC Multiplier = 0x0000000000000000
(XEN) TPR Threshold = 0x00  PostedIntrVec = 0x00
(XEN) EPT pointer = 0x00000002c94eb01e  EPTP index = 0x0000
(XEN) Virtual processor ID = 0x0ee5 VMfunc controls = 0000000000000000
(XEN) **************************************
(XEN) domain_crash called from vmx.c:3360
(XEN) Domain 18 (vcpu#1) crashed on cpu#0:
(XEN) ----[ Xen-4.10.0-3.0-d  x86_64  debug=y   Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    0008:[<0000000082a411dc>]
(XEN) RFLAGS: 0000000000004002   CONTEXT: hvm guest (d18v1)
(XEN) rax: 0000000000000000   rbx: 0000000000000000   rcx: 0000000000000000
(XEN) rdx: 0000000000000000   rsi: 0000000000000000   rdi: 0000000000000000
(XEN) rbp: 0000000000000000   rsp: 00000000807cfba0   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 000000008001003b   cr4: 00000000000406f9
(XEN) cr3: 0000000000185000   cr2: 000000007514d755
(XEN) ds: 0023   es: 0023   fs: 003b   gs: 0000   ss: 0023   cs: 0008

Looking at it some more, the IDTVectoring field reveals what is actually
going on.

We attempted to inject an NMI.  As this is a plain VM without any
hardware, I expect it came from an IPI rather than something external to
the VM.  This also matches up with the interruptibility indicating that
the guest is in an NMI shadow.

Being 32bit, NMIs are handled using a task gate to switch stack, and at
the point of failure, we tried injecting #TS, which suggests that
hvm_task_switch() objected to something.

The error code (0x0010) is odd, but it is certainly possible that
windows puts its TSS adjacent to %cs.  On inspection, there is
definitely a bug in that this error code should have IDT|EXT set in it,
but as this has been broken since forever, its not the source of the
regression.

It is still unclear however which piece of guest state the processor is
actually objecting to.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel