[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] PVH Dom0 Intel IOMMU issues
On Mon, Apr 17, 2017 at 09:03:12AM +0800, Chao Gao wrote: > On Mon, Apr 17, 2017 at 08:47:48AM +0100, Roger Pau Monné wrote: > >On Mon, Apr 17, 2017 at 07:32:45AM +0800, Chao Gao wrote: > >> On Fri, Apr 14, 2017 at 04:34:41PM +0100, Roger Pau Monné wrote: > >> >Hello, > >> > > >> >Although PVHv2 Dom0 is not yet finished, I've been trying the current > >> >code on > >> >different hardware, and found that with pre-Haswell Intel hardware PVHv2 > >> >Dom0 > >> >completely freezes the box when calling iommu_hwdom_init in > >> >dom0_construct_pvh. > >> >OTOH the same doesn't happen when using a newer CPU (ie: haswell or > >> >newer). > >> > > >> >I'm not able to debug that in any meaningful way because the box seems to > >> >lock > >> >up completely, even the watchdog NMI stops working. Here is the boot log, > >> >up to > >> >the point where it freezes: > >> > >> I try "dom0=pvh" with my skylake. An assertion failed. Is it a software > >> bug? > >> > >---8<--- > >diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c > >index 527ac2aadd..1df7710041 100644 > >--- a/xen/arch/x86/hvm/vioapic.c > >+++ b/xen/arch/x86/hvm/vioapic.c > >@@ -625,6 +625,9 @@ int vioapic_init(struct domain *d) > > nr_gsis += nr_pins; > > } > > > >+ printk("domain nr_gsis: %u vioapic gsis: %u nr_irqs_gsi: %u > >highest_gsi: %u\n", > >+ hvm_domain_irq(d)->nr_gsis, nr_gsis, nr_irqs_gsi, highest_gsi()); > >+ > > ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis); > > > > d->arch.hvm_domain.nr_vioapics = nr_vioapics; > > With the above patch, > (XEN) [ 10.420001] PCI: MCFG area at 80000000 reserved in E820 > (XEN) [ 10.426854] PCI: Using MCFG for segment 0000 bus 00-ff > (XEN) [ 10.433952] Intel VT-d iommu 6 supported page sizes: 4kB, 2MB, 1GB. > (XEN) [ 10.441856] Intel VT-d iommu 5 supported page sizes: 4kB, 2MB, 1GB. > (XEN) [ 10.449759] Intel VT-d iommu 4 supported page sizes: 4kB, 2MB, 1GB. > (XEN) [ 10.457671] Intel VT-d iommu 3 supported page sizes: 4kB, 2MB, 1GB. > (XEN) [ 10.465585] Intel VT-d iommu 2 supported page sizes: 4kB, 2MB, 1GB. > (XEN) [ 10.473485] Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB. > (XEN) [ 10.481394] Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB. > (XEN) [ 10.489299] Intel VT-d iommu 7 supported page sizes: 4kB, 2MB, 1GB. > (XEN) [ 10.497196] Intel VT-d Snoop Control enabled. > (XEN) [ 10.503196] Intel VT-d Dom0 DMA Passthrough not enabled. > (XEN) [ 10.510145] Intel VT-d Queued Invalidation enabled. > (XEN) [ 10.516646] Intel VT-d Interrupt Remapping enabled. > (XEN) [ 10.523173] Intel VT-d Posted Interrupt not enabled. > (XEN) [ 10.529775] Intel VT-d Shared EPT tables enabled. > (XEN) [ 10.548815] I/O virtualisation enabled > (XEN) [ 10.554186] - Dom0 mode: Relaxed > (XEN) [ 10.559264] Interrupt remapping enabled > (XEN) [ 10.564854] nr_sockets: 5 > (XEN) [ 10.569231] Enabled directed EOI with ioapic_ack_old on! > (XEN) [ 10.577294] ENABLING IO-APIC IRQs > (XEN) [ 10.582245] -> Using old ACK method > (XEN) [ 10.587967] ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 > (XEN) [ 10.797645] TSC deadline timer enabled > (XEN) [ 10.887286] Defaulting to alternative key handling; send 'A' to > switch to normal mode. > (XEN) [ 10.897864] mwait-idle: MWAIT substates: 0x2020 > (XEN) [ 10.899335] mwait-idle: v0.4.1 model 0x55 > (XEN) [ 10.900799] mwait-idle: lapic_timer_reliable_states 0xffffffff > (XEN) [ 10.902304] VMX: Supported advanced features: > (XEN) [ 10.903781] - APIC MMIO access virtualisation > (XEN) [ 10.905258] - APIC TPR shadow > (XEN) [ 10.907138] - Extended Page Tables (EPT) > (XEN) [ 10.908782] - Virtual-Processor Identifiers (VPID) > (XEN) [ 10.910262] - Virtual NMI > (XEN) [ 10.911719] - MSR direct-access bitmap > (XEN) [ 10.913188] - Unrestricted Guest > (XEN) [ 10.914650] - APIC Register Virtualization > (XEN) [ 10.916126] - Virtual Interrupt Delivery > (XEN) [ 10.917596] - Posted Interrupt Processing > (XEN) [ 10.919066] - VMCS shadowing > (XEN) [ 10.920519] - VM Functions > (XEN) [ 10.921976] - Virtualisation Exceptions > (XEN) [ 10.923448] - Page Modification Logging > (XEN) [ 10.924918] - TSC Scaling > (XEN) [ 10.926371] HVM: ASIDs enabled. > (XEN) [ 10.927829] HVM: VMX enabled > (XEN) [ 10.929278] HVM: Hardware Assisted Paging (HAP) detected > (XEN) [ 10.930762] HVM: HAP page sizes: 4kB, 2MB, 1GB > (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 6, using 0x1 > (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 9, using 0x1 > (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 10, using > 0x1 > (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 11, using > 0x1 > (XEN) [ 13.216648] Brought up 112 CPUs > (XEN) [ 13.739330] build-id: dc4540250abe5d96614d340c67069e390c37c21c > (XEN) [ 13.740816] Running stub recovery selftests... > (XEN) [ 13.742258] traps.c:3466: GPF (0000): ffff82d0bffff041 > [ffff82d0bffff041] -> ffff82d080359cf2 > (XEN) [ 13.745155] traps.c:813: Trap 12: ffff82d0bffff040 > [ffff82d0bffff040] -> ffff82d080359cf2 > (XEN) [ 13.748046] traps.c:1215: Trap 3: ffff82d0bffff041 > [ffff82d0bffff041] -> ffff82d080359cf2 > (XEN) [ 13.771452] TSC warp detected, disabling TSC_RELIABLE > (XEN) [ 13.772897] ACPI sleep modes: S3 > (XEN) [ 13.774329] VPMU: disabled > (XEN) [ 13.775757] mcheck_poll: Machine check polling timer started. > (XEN) [ 13.777246] Dom0 has maximum 1448 PIRQs > (XEN) [ 13.779426] domain nr_gsis: 104 vioapic gsis: 88 nr_irqs_gsi: 104 > highest_gsi: 103 It seems like we are not properly adding/accounting the vIO APICs, but I cannot really see how. I have another patch for you to try below. Thanks, Roger. ---8<--- diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c index 527ac2aadd..40075e2756 100644 --- a/xen/arch/x86/hvm/vioapic.c +++ b/xen/arch/x86/hvm/vioapic.c @@ -610,11 +610,15 @@ int vioapic_init(struct domain *d) xzalloc_array(struct hvm_vioapic *, nr_vioapics)) == NULL) ) return -ENOMEM; + printk("Adding %u vIO APICs\n", nr_vioapics); + for ( i = 0; i < nr_vioapics; i++ ) { unsigned int nr_pins = is_hardware_domain(d) ? nr_ioapic_entries[i] : ARRAY_SIZE(domain_vioapic(d, 0)->domU.redirtbl); + printk("vIO APIC %u has %u pins\n", i, nr_pins); + if ( (domain_vioapic(d, i) = xmalloc_bytes(hvm_vioapic_size(nr_pins))) == NULL ) { @@ -623,8 +627,12 @@ int vioapic_init(struct domain *d) } domain_vioapic(d, i)->nr_pins = nr_pins; nr_gsis += nr_pins; + printk("nr_gsis: %u\n", nr_gsis); } + printk("domain nr_gsis: %u vioapic gsis: %u nr_irqs_gsi: %u highest_gsi: %u\n", + hvm_domain_irq(d)->nr_gsis, nr_gsis, nr_irqs_gsi, highest_gsi()); + ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis); d->arch.hvm_domain.nr_vioapics = nr_vioapics; _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx https://lists.xen.org/xen-devel
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |