Re: BlueIris error on Xen 4.17

[Jan Beulich <jbeulich@xxxxxxxx>]

On 12.06.2024 12:25, Andrew Cooper wrote:
> On 12/06/2024 10:40 am, Damien Thenot wrote:
>> Hello,
>>
>> A XCP-ng 8.3 user that use Blue Iris Software encountered a crash with 
>> Xen upgraded to version 4.17.
>> It worked correctly when XCP-ng 8.3 used Xen 4.13.
>> It is happening on Intel Xeon E-2378 CPU @ 2.60GHz CPUs and it seems 
>> more recent Intel CPUs.
>> His guests are Windows with a NVIDIA GPU given to the guest.
>>
>> The user added:
>>  > On an older box with i9-9900K CPU it does not happen and the VM works 
>> as expected. Also working on an older
>>  > Xeon Intel Xeon E-2146G and a E-2276G. Anything newer than that 
>> however the VM will just BSOD.
>>
>> You can find more information in the XCP-ng forum post: 
>> https://xcp-ng.org/forum/topic/8873/windows-blue-iris-xcp-ng-8-3
>>
>> The user tried enabling `msr-relaxed` following notes in Xen 4.15 
>> documentation.
>> But it didn't change the behavior and the guest still crashes.
>>
>> Has someone else observed such behavior?
>>
>> Here is a Xen dmesg output with the error that the user was able to obtain:
>>
>> ```
>>
>> (d1) [  132.028963] xen|BUGCHECK: ====>
>> (d1) [  132.029008] xen|BUGCHECK: SYSTEM_SERVICE_EXCEPTION: 
>> 00000000C0000096 FFFFF80418A21E27 FFFFAC009F27B900 0000000000000000
>> (d1) [  132.029057] xen|BUGCHECK: EXCEPTION (FFFFF80418A21E27):
>> (d1) [  132.029096] xen|BUGCHECK: - Code = 8589320F
>> (d1) [  132.029134] xen|BUGCHECK: - Flags = 000000A8
>> (d1) [  132.029174] xen|BUGCHECK: - Address = 8589320F008EECE9
>> (d1) [  132.029214] xen|BUGCHECK: - Parameter[0] = 0F000001D9B90000
>> (d1) [  132.029255] xen|BUGCHECK: - Parameter[1] = 4166300FFCE08332
>> (d1) [  132.029297] xen|BUGCHECK: - Parameter[2] = 0355000002C881F7
>> (d1) [  132.029338] xen|BUGCHECK: - Parameter[3] = 0002A0818B497B74
>> (d1) [  132.029379] xen|BUGCHECK: - Parameter[4] = 000002A8918B4900
>> (d1) [  132.029421] xen|BUGCHECK: - Parameter[5] = 8B49CA230FC0230F
>> (d1) [  132.029462] xen|BUGCHECK: - Parameter[6] = 918B49000002B081
>> (d1) [  132.029502] xen|BUGCHECK: - Parameter[7] = 0FD0230F000002B8
>> (d1) [  132.029544] xen|BUGCHECK: - Parameter[8] = 0002C8918B49DA23
>> (d1) [  132.029585] xen|BUGCHECK: - Parameter[9] = 230FF0230FC03300
>> (d1) [  132.029626] xen|BUGCHECK: - Parameter[10] = 008AE22504F665FA
>> (d1) [  132.029667] xen|BUGCHECK: - Parameter[11] = 00C2F76639740200
>> (d1) [  132.029709] xen|BUGCHECK: - Parameter[12] = 81342405F7327403
>> (d1) [  132.029753] xen|BUGCHECK: - Parameter[13] = 6626750000000200
>> (d1) [  132.029794] xen|BUGCHECK: - Parameter[14] = C88303740200C2F7
>> (d1) [  132.029835] xen|BUGCHECK: EXCEPTION (0D8B000000AC9589):
>> (d1) [  132.029888] xen|BUGCHECK: CONTEXT (FFFFAC009F27B900):
>> (d1) [  132.029925] xen|BUGCHECK: - GS = 002B
>> (d1) [  132.029959] xen|BUGCHECK: - FS = 0053
>> (d1) [  132.029994] xen|BUGCHECK: - ES = 002B
>> (d1) [  132.030028] xen|BUGCHECK: - DS = 002B
>> (d1) [  132.030056] xen|BUGCHECK: - SS = 0018
>> (d1) [  132.030089] xen|BUGCHECK: - CS = 0010
>> (d1) [  132.030123] xen|BUGCHECK: - EFLAGS = 00040046
>> (d1) [  132.030160] xen|BUGCHECK: - RDI = 00000000000002C4
>> (d1) [  132.030199] xen|BUGCHECK: - RSI = 00000000005FFA48
>> (d1) [  132.030237] xen|BUGCHECK: - RBX = 00000000426ED080
>> (d1) [  132.030275] xen|BUGCHECK: - RDX = 0000000000000000
>> (d1) [  132.030312] xen|BUGCHECK: - RCX = 00000000000001DD
>> (d1) [  132.030349] xen|BUGCHECK: - RAX = 0000000000000000
>> (d1) [  132.030386] xen|BUGCHECK: - RBP = 00000000E4427520
>> (d1) [  132.030424] xen|BUGCHECK: - RIP = 0000000018A21E27
>> (d1) [  132.030463] xen|BUGCHECK: - RSP = 00000000E4427498
>> (d1) [  132.030504] xen|BUGCHECK: - R8 = 0000000000000000
>> (d1) [  132.030543] xen|BUGCHECK: - R9 = 000000009F25A000
>> (d1) [  132.030580] xen|BUGCHECK: - R10 = 00000000000002C4
>> (d1) [  132.030618] xen|BUGCHECK: - R11 = 0000000000000246
>> (d1) [  132.030657] xen|BUGCHECK: - R12 = 0000000002CEB528
>> (d1) [  132.030696] xen|BUGCHECK: - R13 = 00000000A81E3A80
>> (d1) [  132.030735] xen|BUGCHECK: - R14 = 00000000000002C4
>> (d1) [  132.030775] xen|BUGCHECK: - R15 = 0000000000000000
>> (d1) [  132.030812] xen|BUGCHECK: STACK:
>> (d1) [  132.030858] xen|BUGCHECK: 00000000E44274A0: (00000000426ED080 
>> 00000000005FF898 0000000000000000 0000000000000000) ntoskrnl.exe + 
>> 000000000043667A
>> (d1) [  132.030935] xen|BUGCHECK: 00000000005FFA18: (00000000A74AD76E 
>> 0000000000000000 00000000A81E3A80 000000000000000E) 00007FFBA9BFF4D4
>> (d1) [  132.031010] xen|BUGCHECK: 00000000005FFA20: (0000000000000000 
>> 00000000A81E3A80 000000000000000E 0000000000000003) 00007FFBA74AD76E
>> (d1) [  132.031086] xen|BUGCHECK: 00000000005FFA28: (00000000A81E3A80 
>> 000000000000000E 0000000000000003 00000000005FFB10) 0000000000000000
>> (d1) [  132.031151] xen|BUGCHECK: <====
>> (XEN) [  132.040828] memory_map:remove: dom1 gfn=f60c0 mfn=a3080 nr=4
>> (XEN) [  132.040975] memory_map:remove: dom1 gfn=f5000 mfn=a2000 nr=1000
>> (XEN) [  132.041124] memory_map:remove: dom1 gfn=e0000 mfn=4000000 nr=10000
>> (XEN) [  132.041463] memory_map:remove: dom1 gfn=e8000 mfn=4008000 nr=8000
>> (XEN) [  132.041905] memory_map:remove: dom1 gfn=f8000 mfn=4010000 nr=2000
>> (XEN) [  132.042072] ioport_map:remove: dom1 gport=c100 mport=6000 nr=80
>> ```
> 
> There was a #GP fault accessing MSR 0x1DD

That's LASTINTFROMIP, so ...

> You'll need to investigate how that MSR is supposed to behave on real
> hardware, and see if Xen's behaviour matches or not.

... I'd be inclined to assume our get_model_specific_lbr() isn't up-to-date.
Even with architectural LBRs (which I might guess is what their hardware is
offering, except that then software wouldn't be respecting CPUID output) I
take it that LASTINT{FROM,TO}IP would still be available and working as
before (leaving aside layout aspects).

Jan