Re: Design session "MSI-X support with Linux stubdomain" notes

[Jan Beulich <jbeulich@xxxxxxxx>]

On 22.09.2022 18:05, Anthony PERARD wrote:
> WARNING: Notes missing at the beginning of the meeting.
> 
> session description:
>> Currently a HVM with PCI passthrough and Qemu Linux stubdomain doesn’t
>> support MSI-X. For the device to (partially) work, Qemu needs a patch masking
>> MSI-X from the PCI config space. Some drivers are not happy about that, which
>> is understandable (device natively supports MSI-X, so fallback path are
>> rarely tested).
>>
>> This is mostly (?) about qemu accessing /dev/mem directly (here:
>> https://github.com/qemu/qemu/blob/master/hw/xen/xen_pt_msi.c#L579) - lets
>> discuss alternative interface that stubdomain could use.
> 
> 
> 
> when qemu forward interrupt,
>     for correct mask bit, it read physical mask bit.
>     an hypercall would make sense.
>     -> benefit, mask bit in hardware will be what hypervisor desire, and 
> device model desire.
>     from guest point of view, interrupt should be unmask.
> 
> interrupt request are first forwarded to qemu, so xen have to do some post 
> processing once request comes back from qemu.
>     it's weird..
> 
> someone should have a look, and rationalize this weird path.
> 
> Xen tries to not forward everything to qemu.
> 
> why don't we do that in xen.
>     there's already code in xen for that.

So what I didn't pay enough attention to when talking was that the
completion logic in Xen is for writes only. Maybe something similar
can be had for reads as well, but that's to be checked ...

Jan

> Issue: having QEMU open /dev/mem within stubdom isn't working.
> 
> We could look at removing the need for /dev/mem by improving support for 
> qemu-depriv.
> 
> hypervisor configuration interface was intended for one domain. having 
> stubdom in
> the middle makes thing difficult.
> 
> See QEMU's code
>     https://github.com/qemu/qemu/blob/master/hw/xen/xen_pt_msi.c#L579
>         fd = open("/dev/mem", O_RDWR);
> 
> TODO:
> step1: Find out why qemu wants that mask?
> step2: identify what is missing in the PV interface.
> 
> QEMU use this to read the Pending Bit Array (PBA), and read entry in  table
> 
> comments at L465 (of xen_pt_msi.c) doesn't makes sense
> 
> Xen could do more fixup
> 
> passing value from hardware??
>     can't pass vector to the guest,
>     xen overwrite mask bit. (or something)
> 
> Did MSI-X worked in qemu-trad in stubdom?
>     No one in the room could remember.
> 
> MSI-X is required for pci express, not that thing are implemented correctly.
> 
> TODO:
> - get rid of opening /dev/mem in qemu
> 
> 
> Cheers,
>