|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] Xen-unstable: pci-passthrough "irq 16: nobody cared" on HVM guest shutdown on irq of device not passed through.
> Hi Jan,
>
> Just tested with about the same kernel (and same hardware) on linux + kvm:
>
> When i boot i see (on the host):
> [ 1.341556] pcieport 0000:00:02.0: irq 31 for MSI/MSI-X
> [ 1.341875] pcieport 0000:00:03.0: irq 32 for MSI/MSI-X
> [ 1.342145] pcieport 0000:00:05.0: irq 33 for MSI/MSI-X
> [ 1.342429] pcieport 0000:00:06.0: irq 34 for MSI/MSI-X
> [ 1.342588] pcieport 0000:00:09.0: irq 35 for MSI/MSI-X
> [ 1.342853] pcieport 0000:00:0a.0: irq 36 for MSI/MSI-X
> [ 1.343125] pcieport 0000:00:0b.0: irq 37 for MSI/MSI-X
> [ 1.343285] pcieport 0000:00:0c.0: irq 38 for MSI/MSI-X
> [ 1.343434] pcieport 0000:00:0d.0: irq 39 for MSI/MSI-X
> [ 1.343719] pcieport 0000:00:15.0: irq 40 for MSI/MSI-X
When this happend, did you see:
[ 66.819396] bus: 'pci': really_probe: bound device 0000:00:0d.0 to driver
pcieport
[ 66.842286] bus: 'pci': driver_probe_device: matched device 0000:00:15.0
with driver pcieport
[ 66.868014] bus: 'pci': really_probe: probing driver pcieport with device
0000:00:15.0
-- some other code --
[ 67.757210] pcieport 0000:00:15.0: ?!?!? acpi_pci_irq_enable: PCI INT A ->
GSI 16 (level, low) -> IRQ/rc 16
> [ 1.343897] pcieport 0000:05:00.0: irq 41 for MSI/MSI-X
> [ 1.344134] pcieport 0000:06:01.0: irq 42 for MSI/MSI-X
> [ 1.344398] pcieport 0000:06:02.0: irq 44 for MSI/MSI-X
>
> While the soundcard has still irq16.
>
> Passing through 09:00.0 to a KVM guest (as primary passthrough since that
> works
> with KVM) it works fine and after shutting down there is no "irq 16 nobody
> cared".
>
> So why would linux setup those irq/gsi's for the pcieport differently on Xen
> and on
> baremetal (and why can a irq/gsi be bound to 2 devices) ?
Right, that would be odd.
The IRQ can be bound to two devices if it is an 'level' type (aka, not an
'edge').
This is something that be extrapolated from the ACPI MADT tables
and the xen_register_gsi is the one that is told about that (polarity and
level).
Looking back at your emails:
(XEN) [2014-09-25 13:29:04.111] IRQ: 16 affinity:01 vec:89
type=IO-APIC-level status=00000030 in-flight=0 domain-list=0: 16(--
It definitly is level, so that means it is shared.
Now there is a gotcha with shared interrupts - if there are _two_
devices and one is in one guest while the other is in another guest - both
domains have to Ack it.
That is where Xen PCIBack comes in - it has its own IRQ handler that will
determine (based on a hypercall) whether the device is shared and if so
Ack it. I am wondering if it is turned off (as in, Xen thinks it should not
be shared but Linux has the device shared).
You can fiddle with the Xen pciback parameters to turn on/off this 'fake'
irq handler.
But it all seems to point that we somehow are getting the wrong impression
about the 0000:00:0d.0 and 0000:00:15.0 and using legacy interrupts
instead of MSI ones. And I think that underlaying problem is causing
these secondary ones.
Could you attach also the full dmesg under baremetal with 'debug' and all
kinds of debug enabled ? That should help a bit in figuring out why
they get MSIs under baremetal but legacy interrupts under Xen.
Thank you!
>
> --
> Sander
>
>
> # cat /proc/interrupts
> CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
> 0: 100420 0 0 0 0 0
> IR-IO-APIC-edge timer
> 1: 1 0 0 0 0 2
> IR-IO-APIC-edge i8042
> 7: 1 0 0 0 0 0
> IR-IO-APIC-edge
> 8: 0 0 0 0 0 1
> IR-IO-APIC-edge rtc0
> 9: 0 0 0 0 0 0
> IR-IO-APIC-fasteoi acpi
> 12: 0 0 0 0 0 4
> IR-IO-APIC-edge i8042
> 16: 0 0 0 0 0 796
> IR-IO-APIC 16-fasteoi snd_hda_intel
> 17: 0 0 0 0 0 4
> IR-IO-APIC 17-fasteoi ehci_hcd:usb1, ehci_hcd:usb2, ehci_hcd:usb3
> 18: 0 0 0 0 11 6414
> IR-IO-APIC 18-fasteoi ohci_hcd:usb4, ohci_hcd:usb5, ohci_hcd:usb6,
> ohci_hcd:usb7
> 29: 0 0 0 0 0 0
> IR-PCI-MSI-edge AMD-Vi
> 31: 0 0 0 0 0 0
> IR-PCI-MSI-edge PCIe PME
> 32: 0 0 0 0 0 0
> IR-PCI-MSI-edge PCIe PME
> 33: 0 0 0 0 0 0
> IR-PCI-MSI-edge PCIe PME
> 34: 0 0 0 0 0 0
> IR-PCI-MSI-edge PCIe PME
> 35: 0 0 0 0 0 0
> IR-PCI-MSI-edge PCIe PME
> 36: 0 0 0 0 0 0
> IR-PCI-MSI-edge PCIe PME
> 37: 0 0 0 0 0 0
> IR-PCI-MSI-edge PCIe PME
> 38: 0 0 0 0 0 0
> IR-PCI-MSI-edge PCIe PME
> 39: 0 0 0 0 0 0
> IR-PCI-MSI-edge PCIe PME
> 40: 0 0 0 0 0 0
> IR-PCI-MSI-edge PCIe PME
> 45: 0 0 0 1 79 95630
> IR-PCI-MSI-edge ahci0
> 46: 0 0 0 0 0 0
> IR-PCI-MSI-edge ahci1
> 47: 0 0 0 0 27 8643
> IR-PCI-MSI-edge ahci2
> 48: 0 0 0 0 0 0
> IR-PCI-MSI-edge ahci3
> 49: 0 0 0 0 0 0
> IR-PCI-MSI-edge ahci4
> 50: 0 0 0 0 0 0
> IR-PCI-MSI-edge ahci5
> 53: 0 0 0 0 0 0
> IR-PCI-MSI-edge ahci
> 55: 0 0 0 0 184 22648
> IR-PCI-MSI-edge eth0
> 57: 0 0 0 1 148 21232
> IR-PCI-MSI-edge eth1
> 59: 0 0 0 0 0 1
> IR-PCI-MSI-edge xhci_hcd
> 60: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 61: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 62: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 63: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 64: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 65: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 66: 0 0 0 0 0 23
> IR-PCI-MSI-edge xhci_hcd
> 67: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 68: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 69: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 70: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 71: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 72: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 74: 0 0 0 0 23 22099
> IR-PCI-MSI-edge xhci_hcd
> 75: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 76: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 77: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 78: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 79: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 80: 0 0 0 0 0 0
> IR-PCI-MSI-edge xhci_hcd
> 81: 0 0 0 0 0 0
> IR-IO-APIC 23-fasteoi cx25821[1]
> 83: 0 0 0 0 0 28
> IR-PCI-MSI-edge snd_hda_intel
> 85: 0 0 0 0 3 652
> IR-PCI-MSI-edge vfio-msi[0](0000:09:00.0)
> NMI: 3 3 3 4 4 3
> Non-maskable interrupts
> LOC: 13423 25027 24547 30018 30575 46789
> Local timer interrupts
> SPU: 0 0 0 0 0 0
> Spurious interrupts
> PMI: 3 3 3 4 4 3
> Performance monitoring interrupts
> IWI: 0 0 0 0 1 0 IRQ
> work interrupts
> RTR: 0 0 0 0 0 0
> APIC ICR read retries
> RES: 88652 68934 52849 112934 76728 33317
> Rescheduling interrupts
> CAL: 295 287 327 327 311 200
> Function call interrupts
> TLB: 1165 1232 1122 1376 1248 1160 TLB
> shootdowns
> TRM: 0 0 0 0 0 0
> Thermal event interrupts
> THR: 0 0 0 0 0 0
> Threshold APIC interrupts
> MCE: 0 0 0 0 0 0
> Machine check exceptions
> MCP: 4 4 4 4 4 4
> Machine check polls
> THR: 0 0 0 0 0 0
> Hypervisor callback interrupts
> ERR: 1
> MIS: 0
>
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |