|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Random crash during guest start on x86
Hi, On our CI we have randomly a crash during guest boot on x86. We are running on qemu x86_64 using Xen staging. The crash is happening randomly (something like 1 out of 20 times). This is always happening on the first guest we start, we never got it after first guest was successfully started. Please tell me if you need any other info. Here is the guest kernel log: [ 0.000000] Linux version 5.17.6 (@ip-100-64-206-234.eu-west-1.compute.internal) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #1 SMP PREEMPT Mon Jul 25 14:54:50 UTC 2022 [ 0.000000] Command line: console=hvc0 rdinit=/sbin/init [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Enabled xstate features 0x3, context size is 576 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1520 [ 0.000000] ACPI in unprivileged domain disabled [ 0.000000] Released 0 page(s) [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable [ 0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved [ 0.000000] Xen: [mem 0x0000000000100000-0x000000001fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] DMI not present or invalid. [ 0.000000] Hypervisor detected: Xen PV [ 4.842670] tsc: Fast TSC calibration failed [ 4.842916] tsc: Detected 2799.922 MHz processor [ 4.845741] last_pfn = 0x20000 max_arch_pfn = 0x400000000 [ 4.845871] Disabled [ 4.845953] x86/PAT: MTRRs disabled, skipping PAT initialization too. [ 4.846227] x86/PAT: Configuration [0-7]: WB WT UC- UC WC WP UC UC [ 4.848955] Kernel/User page tables isolation: disabled on XEN PV. [ 5.214349] RAMDISK: [mem 0x03400000-0x04158fff] [ 5.221796] NUMA turned off [ 5.221926] Faking a node at [mem 0x0000000000000000-0x000000001fffffff] [ 5.222712] NODE_DATA(0) allocated [mem 0x1ff17000-0x1ff1afff] [ 5.374020] Zone ranges: [ 5.374146] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 5.374349] DMA32 [mem 0x0000000001000000-0x000000001fffffff] [ 5.374450] Normal empty [ 5.374562] Movable zone start for each node [ 5.374647] Early memory node ranges [ 5.374730] node 0: [mem 0x0000000000001000-0x000000000009ffff] [ 5.376184] node 0: [mem 0x0000000000100000-0x000000001fffffff] [ 5.376586] Initmem setup node 0 [mem 0x0000000000001000-0x000000001fffffff] [ 5.377581] On node 0, zone DMA: 1 pages in unavailable ranges [ 5.378034] On node 0, zone DMA: 96 pages in unavailable ranges [ 5.385508] p2m virtual area at (____ptrval____), size is 40000000 [ 6.156878] Remapped 0 page(s) [ 6.160205] smpboot: Allowing 1 CPUs, 0 hotplug CPUs [ 6.161126] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 6.161230] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000fffff] [ 6.161425] [mem 0x20000000-0xffffffff] available for PCI devices [ 6.161565] Booting kernel on Xen [ 6.161636] Xen version: 4.17-unstable (preserve-AD) [ 6.162085] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 6.179113] setup_percpu: NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:1 nr_node_ids:1 [ 6.200627] percpu: Embedded 53 pages/cpu s179864 r8192 d29032 u2097152 [ 6.204411] Fallback order for Node 0: 0 [ 6.204883] Built 1 zonelists, mobility grouping on. Total pages: 128768 [ 6.204954] Policy zone: DMA32 [ 6.205071] Kernel command line: console=hvc0 rdinit=/sbin/init [ 6.211967] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes, linear) [ 6.214335] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes, linear) [ 6.220677] mem auto-init: stack:off, heap alloc:off, heap free:off [ 6.242643] Memory: 463112K/523900K available (16396K kernel code, 2697K rwdata, 5140K rodata, 1380K init, 1032K bss, 60536K reserved, 0K cma-reserved) [ 6.250830] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 6.292059] Dynamic Preempt: voluntary [ 6.296486] rcu: Preemptible hierarchical RCU implementation. [ 6.296556] rcu: RCU event tracing is enabled. [ 6.296636] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1. [ 6.296805] Trampoline variant of Tasks RCU enabled. [ 6.296954] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 6.297045] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 [ 6.538636] Using NULL legacy PIC [ 6.538760] NR_IRQS: 4352, nr_irqs: 32, preallocated irqs: 0 [ 6.542021] xen:events: Using FIFO-based ABI [ 6.560876] random: get_random_bytes called from start_kernel+0x4a3/0x680 with crng_init=0 [ 6.573619] Console: colour dummy device 80x25 [ 6.575908] printk: console [tty0] enabled [ 6.590639] printk: console [hvc0] enabled [ 6.592790] clocksource: xen: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 6.593865] installing Xen timer for CPU 0 [ 6.599636] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x285bf6ea05e, max_idle_ns: 440795246805 ns [ 6.604322] Calibrating delay loop (skipped), value calculated using timer frequency.. 5599.84 BogoMIPS (lpj=2799922) [ 6.605421] pid_max: default: 32768 minimum: 301 [ 6.607341] LSM: Security Framework initializing [ 6.608878] SELinux: Initializing. [ 6.612563] Mount-cache hash table entries: 1024 (order: 1, 8192 bytes, linear) [ 6.613569] Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes, linear) [ 6.646968] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 6.647269] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 6.647785] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization [ 6.648329] Spectre V2 : Mitigation: Retpolines [ 6.648540] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch [ 6.648854] Speculative Store Bypass: Vulnerable [ 6.649437] MDS: Vulnerable: Clear CPU buffers attempted, no microcode [ 6.679020] general protection fault, maybe for address 0x8800: 0000 [#1] PREEMPT SMP NOPTI [ 6.679020] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.17.6 #1 [ 6.679020] RIP: e030:error_entry+0xaf/0xe0 [ 6.679020] Code: 29 89 c8 48 39 84 24 88 00 00 00 74 15 48 81 bc 24 88 00 00 00 63 10 e0 81 75 03 0f 01 f8 90 90 90 c3 48 89 8c 24 88 00 00 00 <0f> 01 f8 90 90 90 eb 11 0f 20 d8 90 90 90 90 90 48 25 ff e7 ff ff [ 6.679020] RSP: e02b:ffffffff82803a90 EFLAGS: 00000046 [ 6.679020] RAX: 0000000000008800 RBX: 0000000000000000 RCX: ffffffff81e00fa7 [ 6.679020] RDX: 0000000000000000 RSI: ffffffff81e009f8 RDI: 00000000000000eb [ 6.679020] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [ 6.679020] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 [ 6.679020] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 [ 6.679020] FS: 0000000000000000(0000) GS:ffff88801f200000(0000) knlGS:0000000000000000 [ 6.679020] CS: 10000e030 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 6.679020] CR2: 0000000000000000 CR3: 000000000280c000 CR4: 0000000000050660 [ 6.679020] Call Trace: [ 6.679020] <TASK> [ 6.679020] RIP: e030:native_irq_return_iret+0x0/0x2 [ 6.679020] Code: 41 5d 41 5c 5d 5b 41 5b 41 5a 41 59 41 58 58 59 5a 5e 5f 48 83 c4 08 eb 0a 0f 1f 00 90 66 0f 1f 44 00 00 f6 44 24 20 04 75 02 <48> cf 57 0f 01 f8 eb 12 0f 20 df 90 90 90 90 90 48 81 e7 ff e7 ff [ 6.679020] RSP: e02b:ffffffff82803b48 EFLAGS: 00000046 ORIG_RAX: 000000000000e030 [ 6.679020] RAX: 0000000000008800 RBX: ffffffff82803be0 RCX: ffffffff81e00f95 [ 6.679020] RDX: ffffffff81e00f94 RSI: ffffffff81e00f95 RDI: 00000000000000eb [ 6.679020] RBP: 00000000000000eb R08: 0000000090001f0f R09: 0000000000000007 [ 6.679020] R10: ffffffff81e00f94 R11: ffffffff8285a6c0 R12: 0000000000000000 [ 6.679020] R13: ffffffff81e00f94 R14: 0000000000000006 R15: 0000000000000006 [ 6.679020] ? asm_exc_general_protection+0x8/0x30 [ 6.679020] ? restore_regs_and_return_to_kernel+0x1b/0x27 [ 6.679020] ? restore_regs_and_return_to_kernel+0x1b/0x27 [ 6.679020] ? restore_regs_and_return_to_kernel+0x1c/0x27 [ 6.679020] ? restore_regs_and_return_to_kernel+0x1b/0x27 [ 6.679020] ? restore_regs_and_return_to_kernel+0x1c/0x27 [ 6.679020] RIP: e030:insn_get_opcode.part.0+0xab/0x180 [ 6.679020] Code: 00 00 8b 43 4c a9 c0 07 08 00 0f 84 bf 00 00 00 c6 43 1c 01 31 c0 5b 5d c3 83 e2 03 be 01 00 00 00 eb b7 89 ef e8 65 e4 ff ff <89> 43 4c a8 30 75 21 e9 8e 00 00 00 0f b6 7b 03 40 84 ff 75 73 8b [ 6.679020] RSP: e02b:ffffffff82803b70 EFLAGS: 00000246 [ 6.679020] ? restore_regs_and_return_to_kernel+0x1b/0x27 [ 6.679020] insn_get_modrm+0x6c/0x120 [ 6.679020] ? restore_regs_and_return_to_kernel+0x1b/0x27 [ 6.679020] insn_get_sib+0x40/0x80 [ 6.679020] insn_get_displacement+0x82/0x100 [ 6.679020] insn_decode+0xf8/0x100 [ 6.679020] optimize_nops+0x60/0x1e0 [ 6.679020] ? rcu_nmi_exit+0x2b/0x140 [ 6.679020] ? restore_regs_and_return_to_kernel+0x1b/0x27 [ 6.679020] ? native_iret+0x3/0x7 [ 6.679020] ? restore_regs_and_return_to_kernel+0x1c/0x27 [ 6.679020] ? restore_regs_and_return_to_kernel+0x1b/0x27 [ 6.679020] apply_alternatives+0x165/0x2e0 [ 6.679020] ? insn_get_opcode.part.0+0xab/0x180 [ 6.679020] ? insn_get_modrm+0x6c/0x120 [ 6.679020] ? rcu_nmi_enter+0x2b/0xf0 [ 6.679020] ? rcu_nmi_exit+0x2b/0x140 [ 6.679020] ? irqentry_exit+0x29/0x30 [ 6.679020] ? exc_xen_hypervisor_callback+0x8/0x10 [ 6.679020] ? relocate_restore_code+0x22a/0x240 [ 6.679020] alternative_instructions+0x8b/0x13a [ 6.679020] check_bugs+0xa87/0xadf [ 6.679020] ? __get_locked_pte+0xa8/0xf0 [ 6.679020] start_kernel+0x64c/0x680 [ 6.679020] xen_start_kernel+0x592/0x59f [ 6.679020] startup_xen+0x3e/0x3e [ 6.679020] </TASK> [ 6.679020] Modules linked in: [ 6.679020] ---[ end trace 0000000000000000 ]--- [ 6.679020] RIP: e030:error_entry+0xaf/0xe0 [ 6.679020] Code: 29 89 c8 48 39 84 24 88 00 00 00 74 15 48 81 bc 24 88 00 00 00 63 10 e0 81 75 03 0f 01 f8 90 90 90 c3 48 89 8c 24 88 00 00 00 <0f> 01 f8 90 90 90 eb 11 0f 20 d8 90 90 90 90 90 48 25 ff e7 ff ff [ 6.679020] RSP: e02b:ffffffff82803a90 EFLAGS: 00000046 [ 6.679020] RAX: 0000000000008800 RBX: 0000000000000000 RCX: ffffffff81e00fa7 [ 6.679020] RDX: 0000000000000000 RSI: ffffffff81e009f8 RDI: 00000000000000eb [ 6.679020] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 [ 6.679020] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 [ 6.679020] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 [ 6.679020] FS: 0000000000000000(0000) GS:ffff88801f200000(0000) knlGS:0000000000000000 [ 6.679020] CS: 10000e030 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 6.679020] CR2: 0000000000000000 CR3: 000000000280c000 CR4: 0000000000050660 [ 6.679020] Kernel panic - not syncing: Attempted to kill the idle task! Cheers Bertrand
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |