Re: [BUG] Xen BUG at mce_amd.c:100 (mc_ec2type) on Zen 3; host left wedged, no reboot

[Teddy Astie <teddy.astie@xxxxxxxxxx>]

Le 23/10/2026 à 12:20, Peter Deggendorfer a écrit :
> Hey Xen Team,
>
> I'm hitting a reproducible hypervisor panic in the AMD non-fatal MCE
> polling path on a Ryzen 9 5950X (Zen 3). Eight occurrences since
> 2026-09-09, between 12 hours and 2 weeks apart, always CPU 2.
>
> Environment
> -----------
> Xen:   4.20.4-pre (Debian 4.20.3+127-gc42374a105-0+deb13u1), and still
>         present in 4.20.5-pre (Debian 4.20.4+39-gf1e1321c87-0+deb13u1)
> dom0:  Linux 6.12.111+deb13-amd64 (Debian trixie), PV, 32 vCPUs
> CPU:   AMD Family 25 (0x19) Model 33 (0x21) Stepping 2, Ryzen 9 5950X,
>         microcode 0x0a201210
> Cmdline: dom0_mem=4096M,max:4096M ucode=scan console_timestamps=datems
>
> The panic
> ---------
> (XEN) [2026-10-19 08:58:18.727] Xen BUG at arch/x86/cpu/mcheck/ 
> mce_amd.c:100
> (XEN) CPU:    2
> (XEN) RIP:    e008:[<ffff82d04028a650>] mce_amd.c#mc_ec2type.part.0+0/0x10
> (XEN) rbx: 0000000000000445   r12: 0000000000000011   r15: 0000000000000446
> (XEN) Xen code around <ffff82d04028a650>:
> (XEN)  90 90 90 90 90 90 90 90 <0f> 0b 90 90 90 90 90 90 90 90 90 90 90 90
> (XEN) Xen call trace:
> (XEN)    [<ffff82d04028a650>] R mce_amd.c#mc_ec2type.part.0+0/0x10
> (XEN)    [<ffff82d04028a699>] S mc_amd_recoverable_scan+0x39/0x40
> (XEN)    [<ffff82d04028cfdd>] S mcheck_mca_logout+0x42d/0x840
> (XEN)    [<ffff82d04028a35c>] S amd_nonfatal.c#mce_amd_checkregs+0x1c/0x100
> (XEN)    [<ffff82d040240e8c>] S smp_call_function_interrupt+0x4c/0x90
> (XEN)    [<ffff82d0403123d5>] S do_IRQ+0x2a5/0x650
> (XEN)    [<ffff82d040201915>] S common_interrupt+0x135/0x150
> (XEN)    [<ffff82d0402883ca>] S cpu_idle.c#acpi_idle_do_entry+0x8a/0x140
> (XEN)    [<ffff82d0402887cf>] S cpu_idle.c#acpi_processor_idle+0x1ef/0x650
> (XEN)    [<ffff82d040302cf1>] S domain.c#idle_loop+0xc1/0x130
> (XEN) Panic on CPU 2
>
> The outlined mc_ec2type.part.0 contains nothing but the ud2, so this is
> the "unreachable" default branch of mc_ec2type() -- the extended error
> code matches none of MC_EC_TLB_TYPE, MC_EC_MEM_TYPE, MC_EC_BUS_TYPE.
>
> Across the eight occurrences, r12 appears to be the bank index from the
> loop in mcheck_mca_logout(), with rbx == 0x401 + 4 * r12 (MCi_STATUS)
> and r15 == rbx + 1. Three different banks are involved: 10, 17 and 23.
>
> Not a hardware defect
> ---------------------
> - Started within days of upgrading two hosts from Debian bookworm
>    (Xen 4.17) to trixie (Xen 4.20). They had run 3 years on 4.17 with no
>    incident.
> - Three different banks, so not a single failing block.
> - Updating to the current Debian package (4.20.5-pre, built 2026-10-16)
>    changed nothing.
>
> Possibly related: 532988dccc19 ("x86/mcheck: fix mca_allbanks
> updating").
>
> Also possibly related on the Linux side: 0422b07bc4c2 ("x86/mce/amd:
> Filter bogus hardware errors on Zen3 clients"), where bogus MCA values
> on Zen 3 are filtered rather than treated as errors.
> https://ratatoskr.run/stable/2026/04/7143905
>
> Diagnostics / workaround
> ------------------------
> mce_verbosity=verbose yields nothing useful.
> max_cstate=1 made no difference. mce=false is the only thing that stops
> it, and it is what I'm running now.
>
> One more thing: the host does not actually reboot after the panic.
> "Reboot in five seconds" is printed, but the ring buffer continues with
> normal activity minutes later and the remaining 31 CPUs keep running.
> CPU 2 stays wedged, and "xl debug-keys t" then hangs. Afterwards dom0
> and guest vCPUs start losing timer interrupts permanently (RCU stalls
> with a frozen timer-softirq count while dynticks keeps advancing), so
> VMs hang silently hours or days later.
>
>
> I've attached the full xl dmesg log of the last crash, and I'm happy
> to provide more if needed. I can also test patches.
>
> Thanks for your work, and thanks in advance for taking a look.
>
> Peter
Could be related. I just observed a mysterious mce report on my Linux 
system dmesg (Zen 3 CPU as well, Ryzen 7 5700X3D)
> [39230.481809] [Hardware Error]: System Fatal error.
> [39230.481813] [Hardware Error]: CPU:1 (19:21:2) 
> MC12_STATUS[Over|UE|MiscV|AddrV|PCC|SyndV|UECC|Deferred|Poison|Scrub]: 
> 0xffffffffb8616890
> [39230.481823] [Hardware Error]: Error Addr: 0x0000000000000000
> [39230.481825] [Hardware Error]: IPID: 0x0000000000000000, Syndrome: 
> 0x0000000000000000
> [39230.481830] [Hardware Error]: Bank 12 is reserved.
> [39230.481832] [Hardware Error]: cache level: RESV, tx: INSN
It seems to be non-sense, and the system is still up as if nothing 
happened. No #MC got emitted.
Teddy

Attachment: OpenPGP_signature.asc
Description: OpenPGP digital signature