Re: [Xen-devel] Issue with MSI in a HVM domU with several passed through PCI devices

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 26.06.12 at 04:31, Rolu <rolu@xxxxxxxx> wrote:
> On Mon, Jun 25, 2012 at 1:38 PM, Stefano Stabellini 
> <stefano.stabellini@xxxxxxxxxxxxx> wrote:
>> This is what should happen:
>>
>> 1) Linux configures the device with a 0 vector number and the pirq number
>> in the address field;
>>
>> 2) QEMU notices a vector number of 0 and reads the pirq number from the
>> address field, passing it to xc_domain_update_msi_irq;
>>
>> 3) Xen assignes the given pirq to the physical MSI;
>>
>> 4) The guest issues a EVTCHNOP_bind_pirq hypercall;
>>
>> 5) Xen sets the pirq as "IRQ_PT";
>>
>> 6) When Xen tries to inject the MSI into the guest, hvm_domain_use_pirq
>> returns true so Xen calls send_guest_pirq instead.
>>
>>
>> Obviously 6) is not happening. hvm_domain_use_pirq is:
>>
>> is_hvm_domain(d) && pirq && pirq->arch.hvm.emuirq != IRQ_UNBOUND
>>
>> My guess is that emuirq is IRQ_UNBOUND when it should be IRQ_PT (see
>> above).
> 
> This appears to be true. I added logging to hvm_pci_msi_assert in
> xen/drivers/passthrough/io.c and it indicates that
> pirq->arch.hvm.emuirq is -1 (while IRQ_PT is -2) every time right
> before an unsupported delivery mode message.
> 
> I also log pirq->pirq but I found that most of the time I can't find
> this value anywhere else (I'm not sure how to interpret the value,
> though). For example, in my last try:
> 
> * I get an unsupported delivery mode error for pirq->pirq 55, 54 and
> 53. The vast majority are for 54.
> * I have logging in map_domain_emuirq_pirq in xen/arch/x86/irq.c. It
> gets called with pirq 19, 20, 21, 22, 23, 52, 51, 50, 16, 17, 55.
> Never for 54 or 53. It also gets called with pirq=49,emuirq=23 once
> but complains it's already mapped.
> * I have logging in evtchn_bind_pirq in xen/common/event_channel.c. It
> gets called with bind->pirq 16, 17, 51, 55, 49, 29 (twice), 21, 19,
> 22, 52, 48, 47. Also never 54 or 53.
> * map_domain_emuirq_pirq is called from evtchn_bind_pirq for pirq 16, 17, 
> 55.
> * The qemu log mentions pirq 35, 36 and 37
> 
> It seems pirq values don't always mean the same? Is it a coincidence
> that 55 occurs almost everywhere, or is something going wrong with the
> other two values (53 and 54 versus 16 and 17)?
> 
> I have three MSI capable devices passed through to the domU, and I do
> see groups of three distinct pirqs in the data above - just not the
> same ones in every place I look.
> 
>> So maybe the guest is not issuing a EVTCHNOP_bind_pirq hypercall
>> (__startup_pirq doesn't get called), or Xen is erroring out in
>> map_domain_emuirq_pirq.
> 
> evtchn_bind_pirq gets called, though I'm not sure if it is with the right 
> data.
> 
> map_domain_emuirq_pirq always gets past the checks in the top half
> (i.e. up to the line /* do not store emuirq mappings for pt devices
> */), except for one time with pirq=49,emuirq=23 where it finds they
> are already mapped.
> It is called three times with an emuirq of -2, for pirq 16, 17 and 55.
> This implies their info->arch.hvm.emuirq is also set to -2 (haven't
> directly logged that but it's the only assignment there).
> 
> Interestingly, I get an unsupported delivery mode error for pirq 55
> where my logging says pirq->arch.hvm.emuirq is -1, *after*
> map_domain_emuirq_pirq was called for pirq 55 and emuirq -2.

Adding some logging at the kernel side would seem much more
promising to me based on Stefano's analysis yesterday.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel