Re: [PATCH] x86/msi: prevent MSI entry re-writes of the same data

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Wed, Mar 05, 2025 at 11:30:51AM +0100, Jan Beulich wrote:
> On 28.02.2025 12:32, Roger Pau Monne wrote:
> > @@ -191,8 +193,6 @@ void msi_compose_msg(unsigned vector, const cpumask_t 
> > *cpu_mask, struct msi_msg
> >  
> >  static int write_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
> >  {
> > -    entry->msg = *msg;
> > -
> >      if ( iommu_intremap != iommu_intremap_off )
> >      {
> >          int rc;
> > @@ -203,6 +203,20 @@ static int write_msi_msg(struct msi_desc *entry, 
> > struct msi_msg *msg)
> >              return rc;
> >      }
> >  
> > +    /*
> > +     * Avoid updating the MSI entry if the address and data fields haven't
> > +     * changed.  When using interrupt remapping changing the MSI affinity
> > +     * shouldn't change the interrupt remapping table index, and hence the 
> > MSI
> > +     * address and data fields should remain the same.
> > +     */
> > +    if ( entry->msg.address == msg->address && entry->msg.data == 
> > msg->data )
> > +    {
> > +        entry->msg.dest32 = msg->dest32;
> > +        return 0;
> > +    }
> > +
> > +    entry->msg = *msg;
> 
> It is perhaps pure luck that iommu_update_ire_from_msi() doesn't use entry's
> "msg" field, and hence that this re-arrangement is okay. It's unclear to me
> whether going forward this might not bite us.

I've updated the comment in msi_desc to make it clear what the `msg`
field contains.  If iommu_update_ire_from_msi() has a need to fetch
the previous non-translated data and address fields it could always
add an extra field to msi_desc.

> > @@ -1407,7 +1415,9 @@ int pci_restore_msi_state(struct pci_dev *pdev)
> >          }
> >          type = entry->msi_attrib.type;
> >  
> > -        msg = entry->msg;
> > +        msg.dest32 = entry->msg.dest32;
> > +        msi_compose_msg(desc->arch.vector, NULL, &msg);
> > +        entry->msg = (typeof(entry->msg)){};
> >          write_msi_msg(entry, &msg);
> 
> Hmm, this isn't exactly a "restore" then anymore. That said, re-constructing
> the message may even be more correct. Then, however, the question is whether
> passing NULL as msi_compose_msg()'s middle argument is really appropriate. A
> little bit of commentary may be desirable here in any event, also as to need
> to clear entry->msg.

I can add a comment.  Note that as part of the patch a comment is
already added to both the msi_compose_msg() prototype and definition
regarding what passing a NULL cpu_mask implies.

> 
> There's (at least) one place where behavior changes with the change of what
> we store in struct msi_desc's msg field (previously untranslated, now
> translated): dump_msi() wants to use the untranslated form. I fear it can't
> even re-construct some of the data it means to log (without reading from
> the IRTE).

Oh, I've missed dump_msi().  Let me see what I can do there.

Another possibility is for iommu_update_ire_from_msi() to report
whether the IRTE index has changed, and so the MSI fields have been
updated and need propagating to the hardware.

> > --- a/xen/drivers/passthrough/vtd/iommu.c
> > +++ b/xen/drivers/passthrough/vtd/iommu.c
> > @@ -1182,7 +1182,7 @@ static void cf_check dma_msi_end(struct irq_desc 
> > *desc, u8 vector)
> >  static void cf_check dma_msi_set_affinity(
> >      struct irq_desc *desc, const cpumask_t *mask)
> >  {
> > -    struct msi_msg msg;
> > +    struct msi_msg msg = {};
> >      unsigned int dest;
> >      unsigned long flags;
> >      struct vtd_iommu *iommu = desc->action->dev_id;
> 
> Why not a similar transformation as you do in set_msi_affinity(), eliminating
> the local "dest"?

It was more intrusive, but I can certainly do it.

> A change like the one here is likely needed in __hpet_setup_msi_irq(), to
> prevent accidental "uninitialized struct field" warnings.

Hm, won't the struct be fully initialized in that case, by getting
passed a cpu_mask?  I don't mind doing so however.

> hpet_msi_set_affinity() might then also want to use msi_compose_msg(), albeit
> that may also be regarded as an independent change.

Possibly, note that HPET code doesn't use write_msi_msg(), and hence
is only partially affected by the msi_compose_msg() change, but not
the write_msi_msg() one, as it continues to store the untranslated MSI
address and data fields in hpet_event_channel struct.

Thanks, Roger.