Re: [PATCH 3/3] pci/msi: remove pci_msi_ignore_mask

[Bjorn Helgaas <helgaas@xxxxxxxxxx>]

On Mon, Jan 13, 2025 at 11:25:58AM +0100, Roger Pau Monné wrote:
> On Fri, Jan 10, 2025 at 04:30:57PM -0600, Bjorn Helgaas wrote:
> > On Fri, Jan 10, 2025 at 03:01:50PM +0100, Roger Pau Monne wrote:
> > > Setting pci_msi_ignore_mask inhibits the toggling of the mask bit for 
> > > both MSI
> > > and MSI-X entries globally, regardless of the IRQ chip they are using.  
> > > Only
> > > Xen sets the pci_msi_ignore_mask when routing physical interrupts over 
> > > event
> > > channels, to prevent PCI code from attempting to toggle the maskbit, as 
> > > it's
> > > Xen that controls the bit.
> > > 
> > > However, the pci_msi_ignore_mask being global will affect devices that 
> > > use MSI
> > > interrupts but are not routing those interrupts over event channels (not 
> > > using
> > > the Xen pIRQ chip).  One example is devices behind a VMD PCI bridge.  In 
> > > that
> > > scenario the VMD bridge configures MSI(-X) using the normal IRQ chip (the 
> > > pIRQ
> > > one in the Xen case), and devices behind the bridge configure the MSI 
> > > entries
> > > using indexes into the VMD bridge MSI table.  The VMD bridge then 
> > > demultiplexes
> > > such interrupts and delivers to the destination device(s).  Having
> > > pci_msi_ignore_mask set in that scenario prevents (un)masking of MSI 
> > > entries
> > > for devices behind the VMD bridge.
> > > 
> > > Move the signaling of no entry masking into the MSI domain flags, as that
> > > allows setting it on a per-domain basis.  Set it for the Xen MSI domain 
> > > that
> > > uses the pIRQ chip, while leaving it unset for the rest of the cases.
> > > 
> > > Remove pci_msi_ignore_mask at once, since it was only used by Xen code, 
> > > and
> > > with Xen dropping usage the variable is unneeded.
> > > 
> > > This fixes using devices behind a VMD bridge on Xen PV hardware domains.
> > 
> > Wrap to fit in 75 columns.
> > 
> > The first two patches talk about devices behind VMD not being usable
> > for Xen, but this one says they now work.
> 
> Sorry, let me try to clarify:
> 
> Devices behind a VMD bridge are not known to Xen, however that doesn't
> mean Linux cannot use them.  That's what this series achieves.  By
> inhibiting the usage of VMD_FEAT_CAN_BYPASS_MSI_REMAP and the removal
> of the pci_msi_ignore_mask bodge devices behind a VMD bridge do work
> fine when use from a Linux Xen hardware domain.  That's the whole
> point of the series.
> 
> > But this doesn't undo the
> > code changes or comments added by the first two, so the result is a
> > bit confusing (probably because I know nothing about Xen).
> 
> All patches are needed.  There's probably some confusion about devices
> behind a VMD bridge not being known by Xen vs not being usable by
> Linux running under Xen as a hardware domain.
> 
> With all three patches applied devices behind a VMD bridge work under
> Linux with Xen.

There's certainly confusion in *my* mind because I don't know enough
about Xen to understand the subtlety about devices behind VMD not
being known by Xen but still being usable by Linux running under Xen.

Bjorn