|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-devel] xen-unstable fails to boot on a system with Ivy Bridge stepping C0 cpu
Hi,
I need a help with tracking down following issue:
When trying to boot Xen on a system with Ivy Bridge stepping C0 CPU, it is
stuck on CPU initialization.
I've added some tracing to apic writes/reads and traced it so far to sending
INIT IPI.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging detected.
(XEN) Setting warm reset code and vector.
(XEN) apic_wrmsr (0x280,0x0)
(XEN) apic_rdmsr(0x280) = 0x0
(XEN) Asserting INIT.
(XEN) apic_wrmsr (0x300,0x10000c500)
After this ^^^^ write BSP becomes VERY slow and it takes several minutes
till it gets to sending STARTUP IPI:
(XEN) Sending STARTUP #1.
(XEN) apic_wrmsr (0x280,0x0)
(XEN) apic_rdmsr(0x280) = 0x0
(XEN) After apic_write.
(XEN) apic_wrmsr (0x300,0x10000067c)
After this ^^^^^ BSP restores its normal speed until it sends the next INIT IPI
for the next core.
Eventually it hangs on initializing one of the next CPUs, at least this far it
progressed on overnight boot test. However a bare-metal kernel boots without
this
issue, even with pretty ancient versions (I've tried up to 2.6.12).
So question is: what xen is missing in cpu initialization that kernel has?
I've run out of ideas what to check.
Any ideas how to track down and fix issue are welcome. I'm willing to try them
out
while I can get access to system.
Full boot log is bellow.
PS:
I've seen a similar issue on HP Z820 ("sandy bridge" cpu) system, with the only
difference that each CPU initialization took 30-60 seconds and system
eventually booted.
--
Thanks,
Igor
-----------
(XEN) Xen version 4.2-unstable () (gcc version 4.1.2 20080704 (Red Hat 4.1.2-51)) Wed Nov 2 10:39:29 CET
2011******************************************************************
(XEN) Latest ChangeSet: Tue Nov 01 19:03:38 2011 +0000 24050:068d3d55ce6e (XEN) Console output is synchronous.ed OS, 'e' to edit the (XEN) Bootloader: GNU GRUB 0.97'a' to modify the kernel arguments (XEN) Command line: loglvl=all guest_loglvl=all com1=115200,8n1 console=com1 sync_console apic_verbosity=debug (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds (XEN) EDID info not retrieved because no DDC retrieval method detected (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009ac00 (usable) (XEN) 000000000009ac00 - 00000000000a0000 (reserved) (XEN) 00000000000e0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000ad7f2000 (usable) (XEN) 00000000ad7f2000 - 00000000ad7f8000 (reserved) (XEN) 00000000ad7f8000 - 00000000aefe9000 (usable) (XEN) 00000000aefe9000 - 00000000af57a000 (reserved) (XEN) 00000000af57a000 - 00000000af7fa000 (ACPI NVS) (XEN) 00000000af7fa000 - 00000000af7ff000 (ACPI data) (XEN) 00000000af7ff000 - 00000000af800000 (usable) (XEN) 00000000af800000 - 00000000b0000000 (reserved) (XEN) 00000000f8000000 - 00000000fc000000 (reserved) (XEN) 00000000fec00000 - 00000000fec01000 (reserved) (XEN) 00000000fed10000 - 00000000fed14000 (reserved) (XEN) 00000000fed18000 - 00000000fed1a000 (reserved) (XEN) 00000000fed1c000 - 00000000fed20000 (reserved) (XEN) 00000000fee00000 - 00000000fee01000 (reserved) (XEN) 00000000ff900000 - 00000000ffc00000 (reserved) (XEN) 00000000ffd00000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 000000014e000000 (usable) (XEN) ACPI: RSDP 000F0410, 0024 (r2 INTEL) (XEN) ACPI: XSDT AF7FDE18, 0074 (r1 INTEL IVB-PPT 6222004 MSFT 10013) (XEN) ACPI: FACP AF7E3D98, 00F4 (r4 INTEL IVB-PPT 6222004 MSFT 10013) (XEN) ACPI: DSDT AF7C3018, F800 (r2 INTEL IVB-PPT 0 INTL 20110623) (XEN) ACPI: FACS AF7F9E40, 0040 (XEN) ACPI: APIC AF7FCF18, 00CC (r2 INTEL IVB-PPT 6222004 MSFT 10013) (XEN) ACPI: HPET AF7F8F18, 0038 (r1 A M I PCHHPET 6222004 AMI. 3) (XEN) ACPI: SSDT AF7E5018, 10A8 (r1 TrmRef PtidDevc 1000 INTL 20110623) (XEN) ACPI: SSDT AF7E4918, 0411 (r1 AMI PerfTune 1000 INTL 20110623) (XEN) ACPI: MCFG AF7F8E98, 003C (r1 INTEL SNDYBRDG 6222004 MSFT 97) (XEN) ACPI: SSDT AF7F7018, 08CA (r1 PmRef Cpu0Ist 3000 INTL 20110623) (XEN) ACPI: SSDT AF7F6018, 0A2C (r1 PmRef CpuPm 3000 INTL 20110623) (XEN) ACPI: DMAR AF7F9B18, 00B0 (r1 INTEL SNB 1 INTL 1) (XEN) ACPI: SPCR AF7F8D18, 0050 (r1 A M I APTIO 6222004 AMI. 3) (XEN) System RAM: 4047MB (4144632kB) (XEN) No NUMA configuration found (XEN) Faking a node at 0000000000000000-000000014e000000 (XEN) Domain heap initialised (XEN) found SMP MP-table at 000fca30 (XEN) DMI 2.6 present. (XEN) APIC boot state is 'xapic' (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x408 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[404,0], pm1x_evt[400,0] (XEN) ACPI: 32/64X FACS address mismatch in FADT - af7f9e40/0000000000000000, using 32 (XEN) ACPI: wakeup_vec[af7f9e4c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) apic_mem_read(0x20) = 0x0 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) Processor #0 7:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) Processor #1 7:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) Processor #2 7:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) Processor #3 7:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x04] enabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) Processor #4 7:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x05] enabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) Processor #5 7:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x06] enabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) Processor #6 7:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) Processor #7 7:10 APIC version 21 (XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x08] disabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x09] disabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0a] disabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0b] disabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0c] disabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0d] disabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0e] disabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x0f] disabled) (XEN) apic_mem_read(0x30) = 0x1060015 (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000 (XEN) Table is not found! (XEN) Using ACPI (MADT) for SMP configuration information (XEN) SMP: Allowing 16 CPUs (8 hotplug CPUs) (XEN) mapped APIC to ffff82c3ffdfe000 (fee00000) (XEN) apic_mem_read(0x20) = 0x0 (XEN) apic_mem_read(0xd0) = 0x0 (XEN) mapped IOAPIC to ffff82c3ffdfd000 (fec00000) (XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X (XEN) Switched to APIC driver x2apic_cluster. (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 2294.841 MHz processor. (XEN) Initing memory sharing. (XEN) apic_rdmsr(0x20) = 0x0 (XEN) apic_rdmsr(0x20) = 0x0 (XEN) apic_rdmsr(0x20) = 0x0 (XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7 (XEN) apic_rdmsr(0x330) = 0x10000 (XEN) mce_intel.c:1219: MCA Capability: BCAST 1 SER 0 CMCI 1 firstbank 0 extended MCE MSR 0 (XEN) apic_rdmsr(0x2f0) = 0x10000 (XEN) apic_wrmsr (0x2f0,0x100f7) (XEN) apic_rdmsr(0x2f0) = 0x100f7 (XEN) apic_wrmsr (0x2f0,0xf7) (XEN) apic_wrmsr (0x330,0x100fa) (XEN) apic_wrmsr (0x330,0xfa) (XEN) Intel machine check reporting enabled (XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f (XEN) PCI: MCFG area at f8000000 reserved in E820 (XEN) PCI: Using MCFG for segment 0000 bus 00-3f (XEN) Intel VT-d Snoop Control enabled. (XEN) Intel VT-d Dom0 DMA Passthrough not enabled. (XEN) Intel VT-d Queued Invalidation enabled. (XEN) Intel VT-d Interrupt Remapping enabled. (XEN) Intel VT-d Shared EPT tables not enabled. (XEN) I/O virtualisation enabled (XEN) - Dom0 mode: Relaxed (XEN) apic_rdmsr(0x20) = 0x0 (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) Getting VERSION: 1060015 (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) Getting VERSION: 1060015 (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) Enabled directed EOI with ioapic_ack_old on! (XEN) apic_rdmsr(0x20) = 0x0 (XEN) Getting ID: 0 (XEN) apic_rdmsr(0x350) = 0x700 (XEN) Getting LVT0: 700 (XEN) apic_rdmsr(0x360) = 0x400 (XEN) Getting LVT1: 400 (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) apic_rdmsr(0x20) = 0x0 (XEN) apic_rdmsr(0xd0) = 0x1 (XEN) apic_wrmsr (0x80,0x10) (XEN) apic_rdmsr(0x170) = 0x0 (XEN) apic_rdmsr(0x160) = 0x0 (XEN) apic_rdmsr(0x150) = 0x0 (XEN) apic_rdmsr(0x140) = 0x0 (XEN) apic_rdmsr(0x130) = 0x0 (XEN) apic_rdmsr(0x120) = 0x0 (XEN) apic_rdmsr(0x110) = 0x0 (XEN) apic_rdmsr(0x100) = 0x0 (XEN) apic_rdmsr(0xf0) = 0x10f (XEN) Suppress EOI broadcast on CPU#0 (XEN) apic_wrmsr (0xf0,0x11ff) (XEN) apic_rdmsr(0x350) = 0x700 (XEN) enabled ExtINT on CPU#0 (XEN) apic_wrmsr (0x350,0x700) (XEN) apic_wrmsr (0x360,0x400) (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) apic_wrmsr (0x280,0x0) (XEN) apic_rdmsr(0x280) = 0x0 (XEN) apic_wrmsr (0x370,0xfe) (XEN) apic_wrmsr (0x280,0x0) (XEN) apic_rdmsr(0x280) = 0x0 (XEN) apic_rdmsr(0xd0) = 0x1 (XEN) ENABLING IO-APIC IRQs (XEN) -> Using old ACK method (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) init IO_APIC IRQs (XEN) IO-APIC (apicid-pin) 2-0, 2-16, 2-17, 2-18, 2-19, 2-20, 2-21, 2-22, 2-23 not connected. (XEN) apic_wrmsr (0x350,0x10700) (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) number of MP IRQ sources: 15. (XEN) number of IO-APIC #2 registers: 24. (XEN) testing the IO APIC....................... (XEN) IO APIC #2...... (XEN) .... register #00: 02000000 (XEN) ....... : physical APIC id: 02 (XEN) ....... : Delivery Type: 0 (XEN) ....... : LTS : 0 (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) .... register #01: 00170020 (XEN) ....... : max redirection entries: 0017 (XEN) ....... : PRQ implemented: 0 (XEN) ....... : IO APIC version: 0020 (XEN) .... IRQ redirection table: (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: (XEN) 00 000 00 1 0 0 0 0 0 0 00 (XEN) 01 000 00 0 0 0 0 0 1 1 30 (XEN) 02 000 00 0 0 0 0 0 1 1 F0 (XEN) 03 000 00 0 0 0 0 0 1 1 38 (XEN) 04 000 00 0 0 0 0 0 1 1 F1 (XEN) 05 000 00 0 0 0 0 0 1 1 40 (XEN) 06 000 00 0 0 0 0 0 1 1 48 (XEN) 07 000 00 0 0 0 0 0 1 1 50 (XEN) 08 000 00 1 0 0 0 0 1 1 58 (XEN) 09 000 00 1 1 0 0 0 1 1 60 (XEN) 0a 000 00 0 0 0 0 0 1 1 68 (XEN) 0b 000 00 0 0 0 0 0 1 1 70 (XEN) 0c 000 00 0 0 0 0 0 1 1 78 (XEN) 0d 000 00 0 0 0 0 0 1 1 88 (XEN) 0e 000 00 0 0 0 0 0 1 1 90 (XEN) 0f 000 00 0 0 0 0 0 1 1 98 (XEN) 10 000 00 1 0 0 0 0 0 0 00 (XEN) 11 000 00 1 0 0 0 0 0 0 00 (XEN) 12 000 00 1 0 0 0 0 0 0 00 (XEN) 13 000 00 1 0 0 0 0 0 0 00 (XEN) 14 000 00 1 0 0 0 0 0 0 00 (XEN) 15 000 00 1 0 0 0 0 0 0 00 (XEN) 16 000 00 1 0 0 0 0 0 0 00 (XEN) 17 000 00 1 0 0 0 0 0 0 00 (XEN) Using vector-based indexing (XEN) IRQ to pin mappings: (XEN) IRQ240 -> 0:2 (XEN) IRQ48 -> 0:1 (XEN) IRQ56 -> 0:3 (XEN) IRQ241 -> 0:4 (XEN) IRQ64 -> 0:5 (XEN) IRQ72 -> 0:6 (XEN) IRQ80 -> 0:7 (XEN) IRQ88 -> 0:8 (XEN) IRQ96 -> 0:9 (XEN) IRQ104 -> 0:10 (XEN) IRQ112 -> 0:11 (XEN) IRQ120 -> 0:12 (XEN) IRQ136 -> 0:13 (XEN) IRQ144 -> 0:14 (XEN) IRQ152 -> 0:15 (XEN) .................................... done. (XEN) Using local APIC timer interrupts. (XEN) calibrating APIC timer ... (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) apic_wrmsr (0x320,0xf9) (XEN) apic_rdmsr(0x3e0) = 0x0 (XEN) apic_wrmsr (0x3e0,0xb) (XEN) apic_wrmsr (0x380,0x3b9aca00) (XEN) apic_rdmsr(0x390) = 0x3b8c6ca4 (XEN) apic_rdmsr(0x390) = 0x3af42d71 (XEN) ..... CPU clock speed is 2367.0331 MHz. (XEN) ..... host bus clock speed is 99.7765 MHz. (XEN) ..... bus_scale = 0x0000662C (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) apic_wrmsr (0x320,0xf9) (XEN) apic_rdmsr(0x3e0) = 0xb (XEN) apic_wrmsr (0x3e0,0xb) (XEN) apic_wrmsr (0x380,0x0) (XEN) TSC deadline timer enabled (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) apic_wrmsr (0x320,0x400f9) (XEN) apic_rdmsr(0x3e0) = 0xb (XEN) apic_wrmsr (0x3e0,0xb) (XEN) apic_wrmsr (0x380,0x0) (XEN) Platform timer is 14.318MHz HPET (XEN) Allocated console ring of 64 KiB. (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Extended Page Tables (EPT) (XEN) - Virtual-Processor Identifiers (VPID) (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) - Unrestricted Guest (XEN) EPT supports 2MB super page. (XEN) HVM: ASIDs enabled. (XEN) HVM: VMX enabled (XEN) HVM: Hardware Assisted Paging detected. (XEN) Setting warm reset code and vector. (XEN) apic_wrmsr (0x280,0x0) (XEN) apic_rdmsr(0x280) = 0x0 (XEN) Asserting INIT. (XEN) apic_wrmsr (0x300,0x10000c500) (XEN) #startup loops: 2. (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) Sending STARTUP #1. (XEN) apic_wrmsr (0x280,0x0) (XEN) apic_rdmsr(0x280) = 0x0 (XEN) After apic_write. (XEN) apic_wrmsr (0x300,0x10000067c) (XEN) apic_wrmsr (0x280,0x0) (XEN) apic_rdmsr(0x280) = 0x0 (XEN) Sending STARTUP #2. (XEN) apic_wrmsr (0x280,0x0) (XEN) apic_rdmsr(0x280) = 0x0 (XEN) After apic_write. (XEN) apic_wrmsr (0x300,0x10000067c) (XEN) apic_wrmsr (0x280,0x0) (XEN) apic_rdmsr(0x280) = 0x0 (XEN) After Startup. (XEN) After Callout 1. (XEN) Waiting for CALLOUT. (XEN) CALLIN, before setup_local_APIC(). (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) apic_rdmsr(0x20) = 0x1 (XEN) apic_rdmsr(0xd0) = 0x2 (XEN) apic_wrmsr (0x80,0x10) (XEN) apic_rdmsr(0x170) = 0x0 (XEN) apic_rdmsr(0x160) = 0x0 (XEN) apic_rdmsr(0x150) = 0x0 (XEN) apic_rdmsr(0x140) = 0x0 (XEN) apic_rdmsr(0x130) = 0x0 (XEN) apic_rdmsr(0x120) = 0x0 (XEN) apic_rdmsr(0x110) = 0x0 (XEN) apic_rdmsr(0x100) = 0x0 (XEN) apic_rdmsr(0xf0) = 0xff (XEN) Suppress EOI broadcast on CPU#1 (XEN) apic_wrmsr (0xf0,0x11ff) (XEN) apic_rdmsr(0x350) = 0x10000 (XEN) masked ExtINT on CPU#1 (XEN) apic_wrmsr (0x350,0x10700) (XEN) apic_wrmsr (0x360,0x10400) (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) apic_wrmsr (0x280,0x0) (XEN) apic_rdmsr(0x280) = 0x0 (XEN) apic_wrmsr (0x370,0xfe) (XEN) apic_wrmsr (0x280,0x0) (XEN) apic_rdmsr(0x280) = 0x0 (XEN) apic_rdmsr(0xd0) = 0x2 (XEN) apic_rdmsr(0x20) = 0x1 (XEN) apic_rdmsr(0x20) = 0x1 (XEN) apic_rdmsr(0x20) = 0x1 (XEN) apic_rdmsr(0x2f0) = 0x10000 (XEN) apic_wrmsr (0x2f0,0x100f7) (XEN) apic_rdmsr(0x2f0) = 0x100f7 (XEN) apic_wrmsr (0x2f0,0xf7) (XEN) apic_wrmsr (0x330,0x100fa) (XEN) apic_wrmsr (0x330,0xfa) (XEN) OK. (XEN) CPU has booted. (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) apic_wrmsr (0x320,0x400f9) (XEN) apic_rdmsr(0x3e0) = 0x0 (XEN) apic_wrmsr (0x3e0,0xb) (XEN) apic_wrmsr (0x380,0x0) (XEN) apic_wrmsr (0x300,0x2000008fd) (XEN) apic_wrmsr (0x300,0x2000008fd) (XEN) Setting warm reset code and vector. (XEN) apic_wrmsr (0x280,0x0) (XEN) apic_rdmsr(0x280) = 0x0 (XEN) Asserting INIT. (XEN) apic_wrmsr (0x300,0x20000c500) (XEN) #startup loops: 2. (XEN) apic_rdmsr(0x30) = 0x1060015 (XEN) Sending STARTUP #1. (XEN) apic_wrmsr (0x280,0x0) _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |