Re: [BUG] Xen BUG at mce_amd.c:100 (mc_ec2type) on Zen 3; host left wedged, no reboot

[Jan Beulich <jbeulich@xxxxxxxx>]

On 26.10.2026 15:48, Peter Deggendorfer wrote:
> Am 26.10.2026 um 11:53 schrieb Jan Beulich:
>> On 25.10.2026 13:32, Peter Deggendorfer wrote:
>>> Regarding the output of /dev/mcelog: if my decoding is right, three
>>> things stand out. MCi_STATUS is zero, i.e. these banks are reporting
>>> nothing at all -- no error, no garbage value.
>> Well, things can't quite be that way: If MCi_STATUS was zero, the first
>> check in mc_amd_recoverable_scan() would prevent mc_ec2type() from
>> being reached in the first place. Its caller also checks MCi_STATUS_VAL
>> to be set.
>>
>> What I find puzzling is this:
>>
>>      errorcode = status & (MCi_STATUS_MCA | MCi_STATUS_MSEC);
>>      ectype = mc_ec2type(errorcode);
>>
>> With errorcode (and mc_ec2type()'s parameter type) being uint16_t, what
>> use is it to include MCi_STATUS_MSEC in the mask? (Yet that's likely
>> unrelated to the issue at hand.) Considering how mc_ec2type() works, I
>> see no reason why status couldn't be passed to it directly (with the
>> parameter type widened to uint32_t). Then the BUG() there could perhaps
>> become BUG_ON(errorcode).
> 
> You're right, my conclusion was wrong.
> 
> One more data point, from an hour ago:
> 
> (XEN) [2026-10-26 13:13:42.050] MCE: unknown AMD error code 0x1000
> 
> So it isn't always 0. The uint16_t truncation might explain the first
> (MSEC bits dropped), but 0x1000 is a non-zero code that still matches
> none of MC_EC_BUS_TYPE / MC_EC_MEM_TYPE / MC_EC_TLB_TYPE.
> 
> /dev/mcelog right afterwards showed three records, banks 17, 14 and 16,
> all CPU 2 (status zero, as above). Banks seen so far across panics and
> log lines: 10, 14, 16, 17, 23.
> 
> With the BUG() dropped, the host has now logged this twice with no
> panic, no wedged CPU and no hung guests. Everything is running
> normally.
> 
> Regarding passing status as uint32_t and BUG_ON(errorcode): with
> today's 0x1000, bits 0-31 are non-zero but none of the type bits is
> set, so that BUG_ON would still trigger. So unless 0x1000 is a known
> type that mc_ec2type() should handle, I think the poller shouldn't BUG
> on an unclassifiable code at all.

I disagree. That code, aiui, is to decide whether something the hardware
reported as uncorrectable can still be recovered from. Seeing an error
code of unknown type therefore has to be treated as fatal. Whether that
is by keeping that BUG() or by some other means is secondary; continuing
to run Xen is putting all of the system's data at risk, even if in your
particular instance(s) you didn't observe further anomalies.

>>> Both records are again
>>> from CPU 2, and bank 23 is one of the banks that appeared in the
>>> panics, bank 16 is new.
>> Isn't it possible that the lack of 532988dccc19 ("x86/mcheck: fix
>> mca_allbanks updating") simply hid the issue previously?
> 
> That is exactly what I suspect. This host, and 10 more with identical
> hardware, ran for years on Xen 4.17 without a single incident. The
> first panic came within days of upgrading this one to 4.20; the others
> are still on 4.17 and still fine. If no bank was being collected
> before, nothing could have reached mc_ec2type() -- so the condition
> may well have been there the whole time, just never looked at.
> 
> Would it help if I logged the full MCi_STATUS of the first read, i.e.
> in mcheck_mca_logout() right after the MCi_STATUS_VAL check, together
> with CPU and bank number?

Yes please.

Jan