|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] PVH Dom0 Intel IOMMU issues
On Mon, Apr 17, 2017 at 09:03:12AM +0800, Chao Gao wrote:
> On Mon, Apr 17, 2017 at 08:47:48AM +0100, Roger Pau Monné wrote:
> >On Mon, Apr 17, 2017 at 07:32:45AM +0800, Chao Gao wrote:
> >> On Fri, Apr 14, 2017 at 04:34:41PM +0100, Roger Pau Monné wrote:
> >> >Hello,
> >> >
> >> >Although PVHv2 Dom0 is not yet finished, I've been trying the current
> >> >code on
> >> >different hardware, and found that with pre-Haswell Intel hardware PVHv2
> >> >Dom0
> >> >completely freezes the box when calling iommu_hwdom_init in
> >> >dom0_construct_pvh.
> >> >OTOH the same doesn't happen when using a newer CPU (ie: haswell or
> >> >newer).
> >> >
> >> >I'm not able to debug that in any meaningful way because the box seems to
> >> >lock
> >> >up completely, even the watchdog NMI stops working. Here is the boot log,
> >> >up to
> >> >the point where it freezes:
> >>
> >> I try "dom0=pvh" with my skylake. An assertion failed. Is it a software
> >> bug?
> >>
> >---8<---
> >diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
> >index 527ac2aadd..1df7710041 100644
> >--- a/xen/arch/x86/hvm/vioapic.c
> >+++ b/xen/arch/x86/hvm/vioapic.c
> >@@ -625,6 +625,9 @@ int vioapic_init(struct domain *d)
> > nr_gsis += nr_pins;
> > }
> >
> >+ printk("domain nr_gsis: %u vioapic gsis: %u nr_irqs_gsi: %u
> >highest_gsi: %u\n",
> >+ hvm_domain_irq(d)->nr_gsis, nr_gsis, nr_irqs_gsi, highest_gsi());
> >+
> > ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis);
> >
> > d->arch.hvm_domain.nr_vioapics = nr_vioapics;
>
> With the above patch,
> (XEN) [ 10.420001] PCI: MCFG area at 80000000 reserved in E820
> (XEN) [ 10.426854] PCI: Using MCFG for segment 0000 bus 00-ff
> (XEN) [ 10.433952] Intel VT-d iommu 6 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.441856] Intel VT-d iommu 5 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.449759] Intel VT-d iommu 4 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.457671] Intel VT-d iommu 3 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.465585] Intel VT-d iommu 2 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.473485] Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.481394] Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.489299] Intel VT-d iommu 7 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.497196] Intel VT-d Snoop Control enabled.
> (XEN) [ 10.503196] Intel VT-d Dom0 DMA Passthrough not enabled.
> (XEN) [ 10.510145] Intel VT-d Queued Invalidation enabled.
> (XEN) [ 10.516646] Intel VT-d Interrupt Remapping enabled.
> (XEN) [ 10.523173] Intel VT-d Posted Interrupt not enabled.
> (XEN) [ 10.529775] Intel VT-d Shared EPT tables enabled.
> (XEN) [ 10.548815] I/O virtualisation enabled
> (XEN) [ 10.554186] - Dom0 mode: Relaxed
> (XEN) [ 10.559264] Interrupt remapping enabled
> (XEN) [ 10.564854] nr_sockets: 5
> (XEN) [ 10.569231] Enabled directed EOI with ioapic_ack_old on!
> (XEN) [ 10.577294] ENABLING IO-APIC IRQs
> (XEN) [ 10.582245] -> Using old ACK method
> (XEN) [ 10.587967] ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
> (XEN) [ 10.797645] TSC deadline timer enabled
> (XEN) [ 10.887286] Defaulting to alternative key handling; send 'A' to
> switch to normal mode.
> (XEN) [ 10.897864] mwait-idle: MWAIT substates: 0x2020
> (XEN) [ 10.899335] mwait-idle: v0.4.1 model 0x55
> (XEN) [ 10.900799] mwait-idle: lapic_timer_reliable_states 0xffffffff
> (XEN) [ 10.902304] VMX: Supported advanced features:
> (XEN) [ 10.903781] - APIC MMIO access virtualisation
> (XEN) [ 10.905258] - APIC TPR shadow
> (XEN) [ 10.907138] - Extended Page Tables (EPT)
> (XEN) [ 10.908782] - Virtual-Processor Identifiers (VPID)
> (XEN) [ 10.910262] - Virtual NMI
> (XEN) [ 10.911719] - MSR direct-access bitmap
> (XEN) [ 10.913188] - Unrestricted Guest
> (XEN) [ 10.914650] - APIC Register Virtualization
> (XEN) [ 10.916126] - Virtual Interrupt Delivery
> (XEN) [ 10.917596] - Posted Interrupt Processing
> (XEN) [ 10.919066] - VMCS shadowing
> (XEN) [ 10.920519] - VM Functions
> (XEN) [ 10.921976] - Virtualisation Exceptions
> (XEN) [ 10.923448] - Page Modification Logging
> (XEN) [ 10.924918] - TSC Scaling
> (XEN) [ 10.926371] HVM: ASIDs enabled.
> (XEN) [ 10.927829] HVM: VMX enabled
> (XEN) [ 10.929278] HVM: Hardware Assisted Paging (HAP) detected
> (XEN) [ 10.930762] HVM: HAP page sizes: 4kB, 2MB, 1GB
> (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 6, using 0x1
> (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 9, using 0x1
> (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 10, using
> 0x1
> (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 11, using
> 0x1
> (XEN) [ 13.216648] Brought up 112 CPUs
> (XEN) [ 13.739330] build-id: dc4540250abe5d96614d340c67069e390c37c21c
> (XEN) [ 13.740816] Running stub recovery selftests...
> (XEN) [ 13.742258] traps.c:3466: GPF (0000): ffff82d0bffff041
> [ffff82d0bffff041] -> ffff82d080359cf2
> (XEN) [ 13.745155] traps.c:813: Trap 12: ffff82d0bffff040
> [ffff82d0bffff040] -> ffff82d080359cf2
> (XEN) [ 13.748046] traps.c:1215: Trap 3: ffff82d0bffff041
> [ffff82d0bffff041] -> ffff82d080359cf2
> (XEN) [ 13.771452] TSC warp detected, disabling TSC_RELIABLE
> (XEN) [ 13.772897] ACPI sleep modes: S3
> (XEN) [ 13.774329] VPMU: disabled
> (XEN) [ 13.775757] mcheck_poll: Machine check polling timer started.
> (XEN) [ 13.777246] Dom0 has maximum 1448 PIRQs
> (XEN) [ 13.779426] domain nr_gsis: 104 vioapic gsis: 88 nr_irqs_gsi: 104
> highest_gsi: 103
It seems like we are not properly adding/accounting the vIO APICs, but I cannot
really see how. I have another patch for you to try below.
Thanks, Roger.
---8<---
diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
index 527ac2aadd..40075e2756 100644
--- a/xen/arch/x86/hvm/vioapic.c
+++ b/xen/arch/x86/hvm/vioapic.c
@@ -610,11 +610,15 @@ int vioapic_init(struct domain *d)
xzalloc_array(struct hvm_vioapic *, nr_vioapics)) == NULL) )
return -ENOMEM;
+ printk("Adding %u vIO APICs\n", nr_vioapics);
+
for ( i = 0; i < nr_vioapics; i++ )
{
unsigned int nr_pins = is_hardware_domain(d) ? nr_ioapic_entries[i] :
ARRAY_SIZE(domain_vioapic(d, 0)->domU.redirtbl);
+ printk("vIO APIC %u has %u pins\n", i, nr_pins);
+
if ( (domain_vioapic(d, i) =
xmalloc_bytes(hvm_vioapic_size(nr_pins))) == NULL )
{
@@ -623,8 +627,12 @@ int vioapic_init(struct domain *d)
}
domain_vioapic(d, i)->nr_pins = nr_pins;
nr_gsis += nr_pins;
+ printk("nr_gsis: %u\n", nr_gsis);
}
+ printk("domain nr_gsis: %u vioapic gsis: %u nr_irqs_gsi: %u highest_gsi:
%u\n",
+ hvm_domain_irq(d)->nr_gsis, nr_gsis, nr_irqs_gsi, highest_gsi());
+
ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis);
d->arch.hvm_domain.nr_vioapics = nr_vioapics;
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |