Re: Serious AMD-Vi(?) issue

[Jan Beulich <jbeulich@xxxxxxxx>]

On 22.03.2024 20:22, Elliott Mitchell wrote:
> On Fri, Mar 22, 2024 at 04:41:45PM +0000, Kelly Choi wrote:
>>
>> I can see you've recently engaged with our community with some issues you'd
>> like help with.
>> We love the fact you are participating in our project, however, our
>> developers aren't able to help if you do not provide the specific details.
> 
> Please point to specific details which have been omitted.  Fairly little
> data has been provided as fairly little data is available.  The primary
> observation is large numbers of:
> 
> (XEN) AMD-Vi: IO_PAGE_FAULT: DDDD:bb:dd.f d0 addr ffffff???????000 flags 0x8 I
> 
> Lines in Xen's ring buffer.

Yet this is (part of) the problem: By providing only the messages that appear
relevant to you, you imply that you know that no other message is in any way
relevant. That's judgement you'd better leave to people actually trying to
investigate. Unless of course you were proposing an actual code change, with
suitable justification.

In fact when running into trouble, the usual course of action would be to
increase verbosity in both hypervisor and kernel, just to make sure no
potentially relevant message is missed.

>  I recall spotting 3 messages from Linux's
> SATA driver (which weren't saved due to other causes being suspected),
> which would likely be associated with hundreds or thousands of the above
> log messages.  I never observed any messages from the NVMe subsystem
> during that phase.
> 
> The most overt sign was telling the Linux kernel to scan for
> inconsistencies and the kernel finding some.  The domain didn't otherwise
> appear to notice trouble.
> 
> This is from memory, it would take some time to discover whether any
> messages were missed.  Present mitigation action is inhibiting the
> messages, but the trouble is certainly still lurking.

Iirc you were considering whether any of this might be a timing issue. Yet
beyond voicing that suspicion, you didn't provide any technical details as
to why you think so. Such technical details would include taking into
account how IOMMU mappings and associated IOMMU TLB flushing are carried
out. Right now, to me at least, your speculation in this regard fails
basic sanity checking. Therefore the scenario that you're thinking of
would need better describing, imo.

Jan