Re: UBSan bug in real mode fpu emulation

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 24/04/2025 2:08 am, Fabian Specht wrote:
> Dear Xen-Devel team,
>
> we discovered a bug regarding undefined behaviour in the FPU emulation
> unit.
>
> if ( !s->rex_prefix )
> {
>       /* Convert 32-bit real/vm86 to 32-bit prot format. */
>       unsigned int fip = fpstate.env.mode.real.fip_lo +
>                                          (fpstate.env.mode.real.fip_hi << 16);
>       unsigned int fdp = fpstate.env.mode.real.fdp_lo +
>                                          (fpstate.env.mode.real.fdp_hi << 16);
>       unsigned int fop = fpstate.env.mode.real.fop;
>
>       fpstate.env.mode.prot.fip = fip & 0xf;
>       fpstate.env.mode.prot.fcs = fip >> 4;
>       fpstate.env.mode.prot.fop = fop;
>       fpstate.env.mode.prot.fdp = fdp & 0xf;
>       fpstate.env.mode.prot.fds = fdp >> 4;
> }
>
> It occurs at arch/x86/arch/x86/x86_emulate/blk.c:85 of the v4.20.0
> release during the bit shift and can be triggered using the attached xtf
> test. We are not aware of any security consequences.
> Simply shifting by 4 should do the trick in my opinion.
>
> Similar code resides in the same file in lines 87, 125 and 127.
> The attached xtf test is run for hvm32.

Several things.  First, please always the UBSAN analysis from the crash.

There are several different ways that shifts go wrong, and I suspect
this is a shift into a sign bit, which is notable given the unsigned
underlying type.

Also, are you aware that the test isn't properly in Real Mode?  It's in
so-called unreal mode (not actually a real mode, but a consequence of
how the segment registers work), which is relevant to how you manage to
re-enter the emulator for FLDENV.

~Andrew