Re: [Xen-devel] Xen-unstable: pci-passthrough of device using MSI-X interrupts not working after commit x86/MSI: track host and guest masking separately

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 29.06.15 at 13:46, <stefano.stabellini@xxxxxxxxxxxxx> wrote:
> On Fri, 26 Jun 2015, Jan Beulich wrote:
>> >>> On 26.06.15 at 17:04, <JBeulich@xxxxxxxx> wrote:
>> >>>> On 26.06.15 at 15:38, <linux@xxxxxxxxxxxxxx> wrote:
>> >> On 2015-06-26 14:41, Jan Beulich wrote:
>> >>>>>> On 26.06.15 at 13:02, <linux@xxxxxxxxxxxxxx> wrote:
>> >>>> Strange, i don't see *any* of your printk's being hit ... (xl dmesg
>> >>>> attached).
>> >>> 
>> >>> So does the guest (in the working case) use MSI-X at all for the
>> >>> device? I.e. it might be worth comparing the guest's /proc/interrupts
>> >>> from both cases, as the lack of any of the debug messages clearly
>> >>> suggests that such interrupts aren#t being set up.
>> >> 
>> >> In the good case it uses one of them.
>> >> (probably one per port and it has only one usb device connected at 
>> >> present)
>> >> 
>> >> --
>> >> Sander
>> >> 
>> >>           CPU0       CPU1       CPU2       CPU3
>> >>    0:         42          0          0          0   IO-APIC   2-edge      
>> > timer
>> >>[...]
>> >>   83:          8          0          0          0   xen-dyn-event     
>> >> eth0-q3-rx
>> >>   84:       2101          0          0          0  xen-pirq-msi-x    
>> >> xhci_hcd
>> > 
>> > I think this explains it - you're running in PVHVM mode, which I
>> > never tried with those patches. I'd even have to go dig to see how
>> > they drive MSI-X in the first place in that case.
>> 
>> I have an idea: In
>> 
>> static unsigned int startup_msi_irq(struct irq_desc *desc)
>> {
>>     bool_t guest_masked = (desc->status & IRQ_GUEST) &&
>>                           is_hvm_domain(desc->msi_desc->dev->domain);
>> 
>>     if ( unlikely(!msi_set_mask_bit(desc, 0, guest_masked)) )
>>         WARN();
>>     return 0;
>> }
>> 
>> I think we need to also exclude the emuirq case (which is what I
>> understand backs the pvhvm interrupt in the guest - Stefano,
>> please confirm). For testing purposes, could you try simply passing
>> zero instead of guest_masked here?
> 
> emuirq is an irq (or msi) that is generated by an emulated device.
> 
> Obviously a guest cannot tell whether a device is emulated or
> passthrough, as a consequence when the guest remaps irqs (and msis) into
> event channels (arch/x86/pci/xen.c:acpi_register_gsi_xen_hvm and
> arch/x86/pci/xen.c:xen_hvm_setup_msi_irqs), some of the irqs (and msis)
> might actually be emulated.
> 
> Xen uses the concept of "emuirq" to distinguish irqs (and msis) of a
> passthrough device from those belonging to an emulated device. However
> you should know that in a default configuration, without virtio, there
> are no emulated msis. Maybe emuirqs are not really the problem here.
> 
> In the case of passthrough devices that generate MSI-X, the guest remaps
> MSIs into event channels by writing a magic number and the desired pirq
> in the MSI-X table (xen_hvm_setup_msi_irqs), then QEMU retrieves the
> pirq and calls xc_physdev_map_pirq_msi & friends on its behalf.
> Afterwards Linux uses the regular xen_pirq_chip functions to
> enable/disable the pirq. Specifically masking the MSI, is done by
> masking the event channel.
> 
> In the quoted function above, if the issue is finding out whether the
> msi is masked from the guest point of view, then you also need to check
> if it has been remapped into an event channel (you can use
> domain_irq_to_pirq for that), then check its status.

No, the question isn't whether the IRQ is masked from guest's pov,
but whether it should start out masked from the perspective of MSI
emulation code (i.e. whether the guest would subsequently be
expected to unmask it not just by unmasking an associated event
channel). I'll have to check whether domain_irq_to_pirq() can be
used for that at this point in time.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel