[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Xen master hangs



On Mon, Jul 27, 2015 at 4:11 PM, Doug Goldstein <cardoe@xxxxxxxxxx> wrote:
> On Mon, Jul 27, 2015 at 4:55 AM, Andrew Cooper
> <andrew.cooper3@xxxxxxxxxx> wrote:
>> On 24/07/15 17:52, Doug Goldstein wrote:
>>

<snip>

So now I've got a build of staging
8398ec706ff60e17a5044470fa2e90a1b081f37a with debug=y and
frame_pointer=n so that we can get a good call trace. As Andrew
pointed out to me on IRC right now stack tracing is broken with EFI
boots due to a mix of using frame pointers (in Xen) and no frame
pointer (EFI). Here's the latest output:

The system is powered on.
 Xen 4.6-unstable
(XEN) Xen version 4.6-unstable (doug@) (gcc (Ubuntu 4.9.2-10ubuntu13)
4.9.2) debug=y Tue Jul 28 09:03:53 CDT 2015
(XEN) Latest ChangeSet: Fri Jul 24 17:29:41 2015 +0200 git:8398ec7-dirty
(XEN) Bootloader: EFI
(XEN) Command line: com1=115200,8n1,0x40b0 console=com1 loglvl=all
console_timestamp=boot
(XEN) Video information:
(XEN)  VGA is graphics mode 1920x1080, 32 bpp
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) EFI RAM map:
(XEN)  0000000000000000 - 000000000009c000 (usable)
(XEN)  000000000009c000 - 000000000009d000 (reserved)
(XEN)  000000000009d000 - 000000000009e000 (usable)
(XEN)  000000000009e000 - 00000000000c0000 (reserved)
(XEN)  0000000000100000 - 0000000020000000 (usable)
(XEN)  0000000020000000 - 0000000020200000 (reserved)
(XEN)  0000000020200000 - 0000000040004000 (usable)
(XEN)  0000000040004000 - 0000000040005000 (reserved)
(XEN)  0000000040005000 - 00000000ceaad000 (usable)
(XEN)  00000000ceaad000 - 00000000cecaf000 (reserved)
(XEN)  00000000cecaf000 - 00000000d660b000 (usable)
(XEN)  00000000d660b000 - 00000000daa9f000 (reserved)
(XEN)  00000000daa9f000 - 00000000dab9f000 (ACPI NVS)
(XEN)  00000000dab9f000 - 00000000dabff000 (ACPI data)
(XEN)  00000000dabff000 - 00000000dac00000 (usable)
(XEN)  00000000dac00000 - 00000000dfa00000 (reserved)
(XEN)  00000000f80f8000 - 00000000f80f9000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed90000 (reserved)
(XEN)  0000000100000000 - 000000021e600000 (usable)
(XEN)  000000021e600000 - 000000021f000000 (reserved)
(XEN) ACPI: RSDP DABFE014, 0024 (r2 LL      SNB         1 INTL        1)
(XEN) ACPI: UEFI DABD9000, 02A6 (r1 LENOVO TP-G1        2680 PTL         2)
(XEN) ACPI: DBG2 DABD8000, 00E9 (r0 LENOVO TP-G1        2680 PTL         2)
(XEN) ACPI: BGRT DABD7000, 0038 (r0 LENOVO TP-G1        2680 PTL         2)
(XEN) System RAM: 8007MB (8199832kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000021e600000
(XEN) Domain heap initialised
(XEN) SMBIOS 2.7 present.
(XEN) DMI 2.7 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408
(XEN) ACPI: v5 SLEEP INFO: control[1:0], status[1:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:404,1:0], pm1x_evt[1:400,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
dab5a000/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[dab5a00c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 7:10 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
(XEN) Processor #1 7:10 APIC version 21
(XEN) AC 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a301 base: 0xfed00000
(XEN) ERST table was not found
(XEN) ACPI: BGRT: invalidating v0 image at 0xd3649018
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (4 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 760 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) mce_intel.c:735: MCA Capability: BCAST 1 SER 0 CMCI 1 firstbank
0 extended MCE MSR 0
(XEN) Intel machine check reporting enabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2594.177 MHz processor.
(XEN) EFI memory map:
(XEN)  0000000000000-0000000086fff type=7 attr=000000000000000f
(XEN)  0000000087000-0000000087fff type=4 attr=000000000000000f
(XEN)  0000000088000-0000000098fff type=7 attr=000000000000000f
(XEN)  0000000099000-000000009bfff type=2 attr=000000000000000f
(XEN)  000000009c000-000000009cfff type=0 attr=000000000000000f
(XEN)  000000009d000-000000009dfff type=4 attr=000000000000000f
(XEN)  000000009e000-000000009ffff type=0 attr=000000000000000f
(XEN)  0000000100000-000001fffffff type=7 attr=000000000000000f
(XEN)  0000020000000-00000201fffff type=0 attr=000000000000000f
(XEN)  0000020200000-0000040003fff type=7 attr=000000000000000f
(XEN)  0000040004000-0000040004fff type=0 attr=000000000000000f
(XEN)  0000040005000-00000cc48dfff type=7 attr=000000000000000f
(XEN)  00000cc48e000-00000cc4adfff type=4 attr=000000000000000f
(XEN)  00000cc4ae000-00000cc53cfff type=7 attr=000000000000000f
(XEN)  00000cc53d000-00000cea96fff type=2 attr=000000000000000f
(XEN)  00000cea97000-00000ceaacfff type=4 attr=000000000000000f
(XEN)  00000ceaad000-00000cecaefff type=0 attr=000000000000000f
(XEN)  00000cecaf000-00000cf47dfff type=4 attr=000000000000000f
(XEN)  00000cf47e000-00000cf910fff type=7 attr=000000000000000f
(XEN)  00000cf911000-00000d0b10fff type=1 attr=000000000000000f
(XEN)  00000d0b11000-00000d2145fff type=7 attr=000000000000000f
(XEN)  00000d2146000-00000d237ffff type=4 attr=000000000000000f
(XEN)  00000d2380000-00000d2573fff type=7 attr=000000000000000f
(XEN)  00000d2574000-00000d3e52fff type=4 attr=000000000000000f
(XEN)  00000d3e53000-00000d3e56fff type=7 attr=000000000000000f
(XEN)  00000d3e57000-00000d4270fff type=4 attr=000000000000000f
(XEN)  00000d4271000-00000d427afff type=7 attr=000000000000000f
(XEN)  00000d427b000-00000d5c0afff type=4 attr=000000000000000f
(XEN)  00000d5c0b000-00000d6156fff type=7 attr=000000000000000f
(XEN)  00000d6157000-00000d660afff type=3 attr=000000000000000f
(XEN)  00000d660b000-00000d66e0fff type=5 attr=800000000000000f
(XEN)  00000d66e1000-00000d680afff type=5 attr=800000000000000f
(XEN)  00000d680b000-00000d73b3fff type=6 attr=800000000000000f
(XEN)  00000d73b4000-00000da259fff type=6 attr=800000000000000f
(XEN)  00000da25a000-00000da7effff type=0 attr=000000000000000f
(XEN)  00000da7f0000-00000daa9afff type=0 attr=000000000000000f
(XEN)  00000daa9b000-00000daa9bfff type=0 attr=000000000000000f
(XEN)  00000daa9c000-00000daa9efff type=0 attr=000000000000000f
(XEN)  00000daa9f000-00000daaf4fff type=10 attr=000000000000000f
(XEN)  00000daaf5000-00000dab9efff type=10 attr=000000000000000f
(XEN)  00000dab9f000-00000dabd6fff type=9 attr=000000000000000f
(XEN)  0000000000000f
(XEN)  00000dabff000-00000dabfffff type=4 attr=000000000000000f
(XEN)  0000100000000-000021e5fffff type=7 attr=000000000000000f
(XEN)  00000000a0000-00000000bffff type=0 attr=0000000000000000
(XEN)  00000dac00000-00000df9fffff type=0 attr=0000000000000000
(XEN)  00000f80f8000-00000f80f8fff type=11 attr=8000000000000001
(XEN)  00000fed1c000-00000fed1ffff type=11 attr=8000000000000001
(XEN)  00000fed20000-00000fed8ffff type=0 attr=0000000000000000
(XEN)  000021e600000-000021effffff type=0 attr=0000000000000000
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802f6150 -> ffff82d0802f7440
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC deadline timer enabled
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x21120
(XEN) mwait-idle: v0.4 model 0x3a
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB
(XEN) Brought up 4 CPUs
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) Dom0 has maximum 600 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0x1000000 memsz=0xbac000
(XEN) elf_parse_binary: phdr: paddr=0x1c00000 memsz=0x136000
(XEN) elf_parse_binary: phdr: paddr=0x1d36000 memsz=0x15440
(XEN) elf_parse_binary: phdr: paddr=0x1d4c000 memsz=0x2bf000
(XEN) elf_parse_binary: memory: 0x1000000 -> 0x200b000
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff81d4c1f0
(XEN) eck: addresses:
(XEN)     virt_base        = 0xffffffff80000000
(XEN)     elf_paddr_offset = 0x0
(XEN)     virt_offset      = 0xffffffff80000000
(XEN)     virt_kstart      = 0xffffffff81000000
(XEN)     virt_kend        = 0xffffffff8200b000
(XEN)     virt_entry       = 0xffffffff81d4c1f0
(XEN)     p2m_base         = 0xffffffffffffffff
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x200b000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000210000000->0000000214000000 (1963504
pages to be allocated)
(XEN)  Init. ramdisk: 000000021c6f6000->000000021e5ffd57
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff8200b000
(XEN)  Init. ramdisk: 0000000000000000->0000000000000000
(XEN)  Phys-Mach map: ffffffff8200b000->ffffffff82f357d0
(XEN)  Start info:    ffffffff82f36000->ffffffff82f364b4
(XEN)  Page tables:   ffffffff82f37000->ffffffff82f52000
(XEN)  Boot stack:    ffffffff82f52000->ffffffff82f53000
(XEN)  TOTAL:         ffffffff80000000->ffffff(XEN) traps.c:2686:d0v0
Domain attempted WRMSR 00000000c0000081 from 0xe023e00800000000 to
0x0023001000000000.
(XEN) traps.c:2686:d0v0 Domain attempted WRMSR 00000000c0000082 from
0xffff82d0bffff000 to 0xffffffff817cb550.
(XEN) traps.c:2686:d0v0 Domain attempted WRMSR 00000000c0000083 from
0xffff82d0bffff020 to 0xffffffff817cdb80.
(XEN) traps.c:2686:d0v0 Domain attempted WRMSR 0000000000000174 from
0x000000000000e008 to 0x0000000000000010.
(XEN) traps.c:2686:d0v0 Domain attempted WRMSR 0000000000000175 from
0xffff8300cfd07fc0 to 0x0000000000000000.
(XEN) traps.c:2686:d0v0 Domain attempted WRMSR 0000000000000176 from
0xffff82d08023ad70 to 0xffffffff817cde90.
(XEN) traps.c:2686:d0v0 Domain attempted WRMSR 00000000c0000083 from
0xffff82d0bffff020 to 0xffffffff817ce0d0.
(XEN) traps.c:2686:d0v0 Domain attempted WRMSR 00000000c0000084 from
0x0000000000074700 to 0x0000000000047700.
(XEN) traps.c:2686:d0v0 Domain attempted WRMSR 00000000000001c9 from
0x0000000000000000 to 0x0000000000000003.
(XEN) traps.c:2686:d0v0 Domain attempted WRMSR 00000000000001a6 from
0x0000000000000000 to 0x00000000000001ff.
(XEN) traps.c:2686:d0v0 Domain attempted WRMSR 00000000000001a7 from
0x0000000000000000 to 0x00000000000001ff.
(XEN) traps.c:2686:d0v0 Domain attempted WRMSR 00000000000003f6 from
0x000000000000ffff to 0x000000000000fe00.
(XEN) traps.c:2686:d0v1 Domain attempted WRMSR 00000000c0000081 from
0xe023e00800000000 to 0x0023001000000000.
(XEN) traps.c:2686:d0v1 Domain attempted WRMSR 00000000c0000082 from
0xffff82d0bfffe080 to 0xffffffff817cb550.
(XEN) traps.c:2686:d0v1 Domain attempted WRMSR 00000000c0000083 from
0xffff82d0bfffe0a0 to 0xffffffff817cdb80.
(XEN) traps.c:2686:d0v1 Domain attempted WRMSR 0000000000000174 from
0x000000000000e008 to 0x0000000000000010.
(XEN) traps.c:2686:d0v1 Domain attempted WRMSR
00000000000001700000e008 to 0x0000000000000010.
(XEN) traps.c:2686:d0v2 Domain attempted WRMSR 0000000000000175 from
0xffff830216cc7fc0 to 0x0000000000000000.
(XEN) traps.c:2686:d0v2 Domain attempted WRMSR 0000000000000176 from
0xffff82d08023ad70 to 0xffffffff817cde90.
(XEN) traps.c:2686:d0v2 Domain attempted WRMSR 00000000c0000083 from
0xffff82d0bfffd120 to 0xffffffff817ce0d0.
(XEN) traps.c:2686:d0v2 Domain attempted WRMSR 00000000c0000084 from
0x0000000000074700 to 0x0000000000047700.
(XEN) traps.c:2686:d0v3 Domain attempted WRMSR 00000000c0000081 from
0xe023e00800000000 to 0x0023001000000000.
(XEN) traps.c:2686:d0v3 Domain attempted WRMSR 00000000c0000082 from
0xffff82d0bfffc180 to 0xffffffff817cb550.
(XEN) traps.c:2686:d0v3 Domain attempted WRMSR 00000000c0000083 from
0xffff82d0bfffc1a0 to 0xffffffff817cdb80.
(XEN) traps.c:2686:d0v3 Domain attempted WRMSR 0000000000000174 from
0x000000000000e008 to 0x0000000000000010.
(XEN) traps.c:2686:d0v3 Domain attempted WRMSR 0000000000000175 from
0xffff830216c7ffc0 to 0x0000000000000000.
(XEN) traps.c:2686:d0v3 Domain attemp(XEN) d0: Forcing read-only
access to MFN fed00
(XEN) ----[ Xen-4.6-unstable  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    3
(XEN) RIP:    e008:[<00000000cea9727b>] 00000000cea9727b
(XEN) RFLAGS: 0000000000010286   CONTEXT: hypervisor (d0v3)
(XEN) rax: 00000000cea9727b   rbx: ffff830216c7fe48   rcx: 000000000000001f
(XEN) rdx: 00000000d674ded0   rsi: 000000319697ec80   rdi: ffff83021df64080
(XEN) rbp: ffff830216c7fda8   rsp: ffff830216c7fd20   r8:  ffff830216c7fe68
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 00000000db002700
(XEN) r12: ffff830216c7fe68   r13: 0000000000000000   r14: ffff83021df64080
(XEN) r15: 0000000211c13000   cr0: 0000000080050033   cr4: 00000000001526e0
(XEN) cr3: 0000000216cb7000   cr2: 00000000cea9727b
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from rsp=ffff830216c7fd20:
(XEN)    00000000d674cd77 0000000211c13000 ffffffff81ce0a20 ffff82d0802aabc4
(XEN)    ffff83021df64080 8000000000000013 00000000ffffffa1 0000000216cb7000
(XEN)    ffff82d08023f5a1 ffff830216c7fe48 ffff830216c7fde8 000000319697ec80
(XEN)    ffff82d08023f56c ffff830216cc8340 0000000000000003 ffff830216c7fde8
(XEN)    0000000000000206 0000000000000400 0000000000000292 ffff830128e0dd80
(XEN)    ffff830216c7ff18 ffffffff81ce0a20 ffff82d0802aabc4 ffff830216c78000
(XEN)    ffff82d080320080 ffff830216c7fef8 ffff82d0801673ba ffff830216cc8108
(XEN)    0027b02880108237 0000000000000000 ffff880209de7d18 0000000000000002
(XEN)    ffff830216c7fe38 ffff82d08012ce6a ffff830216c7fe58 ffff82d08018ad0c
(XEN)    0300000100000031 0000000000000008 0000000000000000 0000000000000400
(XEN)    ffff8802038c0c00 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 ffff830216c7fef8 ffff8300d65fd000 ffffffff81ce0a20
(XEN)    0000000000000000 ffffffff8167e290 0000000000000000 ffff880209de7da8
(XEN)    ffff82d08023abda ffffffff810010ea 0000000000000007 ffff8802038a85bc
(XEN)    ffff8802038a83dc ffff8802038a85be 0000000000000700 ffff880209de7da8
(XEN)    ffff8802038c0c00 0000000000000246 ffff88020f0d77c0 ffff880209de7de8
(XEN)    00000000000175a0 0000000000000007 ffffffff810010ea 0000000000000000
(XEN)    ffff8802038c0c00 ffff880209de7d18 0001010000000000 ffffffff810010ea
(XEN) Xen call trace:
(XEN)    [<00000000cea9727b>] 00000000cea9727b
(XEN)    [<ffff82d08023f5a1>] efi_runtime_call+0x64e/0x80a
(XEN)    [<ffff82d08023f56c>] efi_runtime_call+0x619/0x80a
(XEN)    [<ffff82d0801673ba>] do_platform_op+0xb76/0x1a14
(XEN)    [<ffff82d08012ce6a>] _spin_lock+0x11/0x52
(XEN)    [<ffff82d08018ad0c>] stime2tsc+0x78/0x82
(XEN)    [<ffff82d08023abda>] lstar_enter+0xda/0x134
(XEN)
(XEN) Pagetable walk from 00000000cea9727b:
(XEN)  L4[0x000] = 0000000216cb6063 ffffffffffffffff
(XEN)  L3[0x003] = 00000000cfca4063 ffffffffffffffff
(XEN)  L2[0x075] = 00000000ce9ff063 ffffffffffffffff
(XEN)  L1[0x097] = 80000000cea97163 00000000000d15e5
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 3:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0011]
(XEN) Faulting linear address: 00000000cea9727b
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

--
Doug Goldstein

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.