[Xen-devel] Real-mode bug with AMD, gPXE, and 32-bit rep movs

[George Dunlap <George.Dunlap@xxxxxxxxxxxxx>]

With recent builds of -unstable, I've been unable to get an HVM domain
to boot on an AMD box with the virtual network card enabled.  The same
exact binaries work on an intel box just fine.

The problem turns out to be with the handling of a 32-bit rep MOVS
instruction in the 16-bit gPXE initialization code.  The offending
code is here:

f3 67 a4 66 89 3e 7b 02

nasm interprets it this way:

00000000  F3                db 0xF3
00000001  67A4              a32 movsb
00000003  66893E7B02        a32 mov [0x27b],edi

In other words, a 32-bit rep movs in 16-bit mode.  (gPXE appeas to be
copying itself from where it was installed at 0xc9000 to somewhere
higher in memory, 0x200000.  Not clear why it wants to do that.)

On Intel boxes, the code causes a #GP (not surprisingly), and the
emulator handles it successfully.

On AMD boxes (at least two of them), this causes a #GP (surprisingly).
That calls to the BIOS "null trap handler", which simply does an iret,
causing a busy loop.

There are three possibilities I came up with:
1) The same thing would happen outside of SVM; in which case it's
(sort of) a gPXE bug for using an instruction that won't work on AMD
boxes.
2) Xen is subtly screwing up the VM state, causing the AMD hardware
not to recognize that this shouldn't cause a #GP
3) AMD hardware (at least some of it) doesn't handle 32-bit rep movs
instructions in 16-bit mode.

If it's #1, we should try to build gPXE without the 32-bit instructions

If it's #2, we need to track down what state is being corrupted by Xen.

If it's #3, the simplest solution is probably to take vmexits on GP
faults and attempt to emulate the instruction if we're in real mode,
as we do for vmx.

Wei, Christoph: any ideas?

The cpuid output of the two boxes I've tried this on is below.

Thanks,
 -George Dunlap

[elite]
processor       : 0

vendor_id       : AuthenticAMD

cpu family      : 16

model           : 2

model name      : Quad-Core AMD Opteron(tm) Processor 2352

stepping        : 3

cpu MHz         : 2094.850

cache size      : 512 KB

fdiv_bug        : no

hlt_bug         : no

f00f_bug        : no

coma_bug        : no

fpu             : yes

fpu_exception   : yes

cpuid level     : 5

wp              : yes

flags           : fpu de tsc msr pae cx8 apic mtrr cmov pat clflush mmx fxsr
sse sse2 ht syscall nx mmxext fxsr_opt 3dnowext 3dnow constant_tsc pni
cmp_legacy cr8legacy ts ttp tm stc [6] [7] [8]

bogomips        : 4190.72


[dakota]processor       : 0

vendor_id       : AuthenticAMD

cpu family      : 15

model           : 65

model name      : Dual-Core AMD Opteron(tm) Processor 2218

stepping        : 2

cpu MHz         : 2593.560

cache size      : 1024 KB

fdiv_bug        : no

hlt_bug         : no

f00f_bug        : no

coma_bug        : no

fpu             : yes

fpu_exception   : yes

cpuid level     : 1

wp              : yes

flags           : fpu de tsc msr pae mce cx8 apic mtrr mca cmov pat clflush mmx
fxsr sse sse2 ht nx mmxext fxsr_opt 3dnowext 3dnow pni cmp_legacy
cr8legacy ts fid vid ttp tm stc

bogomips        : 5188.45

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel