[BUG] Xen BUG at mce_amd.c:100 (mc_ec2type) on Zen 3; host left wedged, no reboot

[Peter Deggendorfer <pd@xxxxxxxxxxxxxxx>]

Hey Xen Team,

I'm hitting a reproducible hypervisor panic in the AMD non-fatal MCE
polling path on a Ryzen 9 5950X (Zen 3). Eight occurrences since
2026-09-09, between 12 hours and 2 weeks apart, always CPU 2.

Environment
-----------
Xen:   4.20.4-pre (Debian 4.20.3+127-gc42374a105-0+deb13u1), and still
       present in 4.20.5-pre (Debian 4.20.4+39-gf1e1321c87-0+deb13u1)
dom0:  Linux 6.12.111+deb13-amd64 (Debian trixie), PV, 32 vCPUs
CPU:   AMD Family 25 (0x19) Model 33 (0x21) Stepping 2, Ryzen 9 5950X,
       microcode 0x0a201210
Cmdline: dom0_mem=4096M,max:4096M ucode=scan console_timestamps=datems

The panic
---------
(XEN) [2026-10-19 08:58:18.727] Xen BUG at arch/x86/cpu/mcheck/mce_amd.c:100
(XEN) CPU:    2
(XEN) RIP:    e008:[<ffff82d04028a650>] mce_amd.c#mc_ec2type.part.0+0/0x10
(XEN) rbx: 0000000000000445   r12: 0000000000000011   r15: 0000000000000446
(XEN) Xen code around <ffff82d04028a650>:
(XEN)  90 90 90 90 90 90 90 90 <0f> 0b 90 90 90 90 90 90 90 90 90 90 90 90
(XEN) Xen call trace:
(XEN)    [<ffff82d04028a650>] R mce_amd.c#mc_ec2type.part.0+0/0x10
(XEN)    [<ffff82d04028a699>] S mc_amd_recoverable_scan+0x39/0x40
(XEN)    [<ffff82d04028cfdd>] S mcheck_mca_logout+0x42d/0x840
(XEN)    [<ffff82d04028a35c>] S amd_nonfatal.c#mce_amd_checkregs+0x1c/0x100
(XEN)    [<ffff82d040240e8c>] S smp_call_function_interrupt+0x4c/0x90
(XEN)    [<ffff82d0403123d5>] S do_IRQ+0x2a5/0x650
(XEN)    [<ffff82d040201915>] S common_interrupt+0x135/0x150
(XEN)    [<ffff82d0402883ca>] S cpu_idle.c#acpi_idle_do_entry+0x8a/0x140
(XEN)    [<ffff82d0402887cf>] S cpu_idle.c#acpi_processor_idle+0x1ef/0x650
(XEN)    [<ffff82d040302cf1>] S domain.c#idle_loop+0xc1/0x130
(XEN) Panic on CPU 2

The outlined mc_ec2type.part.0 contains nothing but the ud2, so this is
the "unreachable" default branch of mc_ec2type() -- the extended error
code matches none of MC_EC_TLB_TYPE, MC_EC_MEM_TYPE, MC_EC_BUS_TYPE.

Across the eight occurrences, r12 appears to be the bank index from the
loop in mcheck_mca_logout(), with rbx == 0x401 + 4 * r12 (MCi_STATUS)
and r15 == rbx + 1. Three different banks are involved: 10, 17 and 23.

Not a hardware defect
---------------------
- Started within days of upgrading two hosts from Debian bookworm
  (Xen 4.17) to trixie (Xen 4.20). They had run 3 years on 4.17 with no
  incident.
- Three different banks, so not a single failing block.
- Updating to the current Debian package (4.20.5-pre, built 2026-10-16)
  changed nothing.

Possibly related: 532988dccc19 ("x86/mcheck: fix mca_allbanks
updating").

Also possibly related on the Linux side: 0422b07bc4c2 ("x86/mce/amd:
Filter bogus hardware errors on Zen3 clients"), where bogus MCA values
on Zen 3 are filtered rather than treated as errors.
https://ratatoskr.run/stable/2026/04/7143905

Diagnostics / workaround
------------------------
mce_verbosity=verbose yields nothing useful.
max_cstate=1 made no difference. mce=false is the only thing that stops
it, and it is what I'm running now.

One more thing: the host does not actually reboot after the panic.
"Reboot in five seconds" is printed, but the ring buffer continues with
normal activity minutes later and the remaining 31 CPUs keep running.
CPU 2 stays wedged, and "xl debug-keys t" then hangs. Afterwards dom0
and guest vCPUs start losing timer interrupts permanently (RCU stalls
with a frozen timer-softirq count while dynticks keeps advancing), so
VMs hang silently hours or days later.


I've attached the full xl dmesg log of the last crash, and I'm happy
to provide more if needed. I can also test patches.

Thanks for your work, and thanks in advance for taking a look.

Peter

Attachment: xl_dmesg.log
Description: Text document