Re: [Xen-devel] [PATCH RFC] pass-through: sync pir to irr after msix vector been updated

[Joe Jin <joe.jin@xxxxxxxxxx>]

On 9/13/19 3:33 AM, Roger Pau Monné wrote:
> On Thu, Sep 12, 2019 at 11:03:14AM -0700, Joe Jin wrote:
>> With below testcase, guest kernel reported "No irq handler for vector":
>>   1). Passthrough mlx ib VF to 2 pvhvm guests.
>>   2). Start rds-stress between 2 guests.
>>   3). Scale down 2 guests vcpu from 32 to 6 at the same time.
>>
>> Repeat above test several iteration, guest kernel reported "No irq handler
>> for vector", and IB traffic downed to zero which caused by interrupt lost.
>>
>> When vcpu offline, kernel disabled local IRQ, migrate IRQ to other cpu,
>> update MSI-X table, enable IRQ. If any new interrupt arrived after
>> local IRQ disabled also before MSI-X table been updated, interrupt still 
>> used old vector and dest cpu info, and when local IRQ enabled again, 
>> interrupt been sent to wrong cpu and vector.
> 
> Yes, but that's something Linux shoulkd be able to handle, according
> to your description there's a window where interrupts can be delivered
> to the old CPU, but that's something expected.

Actually, kernel will check APIC IRR when do migration, if any pending
IRQ, will retrigger IRQ to new destination, but Xen does not set the
bit.

> 
>>
>> Looks sync PIR to IRR after MSI-X been updated is help for this issue.
> 
> AFAICT the sync that you do is still using the old vcpu id, as
> pirq_dpci->gmsi.dest_vcpu_id gets updated a little bit below. I'm
> unsure about why does this help, I would expect the sync between pir
> and irr to happen anyway, and hence I'm not sure why is this helping.

As my above update, IRQ retriggered by old cpu, so Xen need to set IRR
for old cpu but not of new.

Thanks,
Joe



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel