Xen project Mailing List

Re: [Xen-devel] pv-grub boot hangs when iommu=soft. Booting kernel directly with the flags works. MiniOS screws around with page flags.

To: Bruce Edge <bruce.edge@xxxxxxxxx>

From: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>

Date: Tue, 6 Jul 2010 12:42:56 -0400

Cc: Samuel Thibault <samuel.thibault@xxxxxxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx, listmail <listmail@xxxxxxxxxxxx>

Delivery-date: Tue, 06 Jul 2010 09:45:32 -0700

List-id: Xen developer discussion <xen-devel.lists.xensource.com>

On Tue, Jul 06, 2010 at 09:06:26AM -0700, Bruce Edge wrote: > On Wed, Apr 7, 2010 at 9:57 AM, Samuel Thibault < > samuel.thibault@xxxxxxxxxxxx> wrote: > > > listmail, le Wed 07 Apr 2010 12:53:19 -0400, a écrit : > > > "If the 'iommu=soft swiotlb=force' is removed it boots" > > > > > > but in actuality, "switotlb=force" can be left in and it boots ok as > > well. > > > From what I've see it is only when having "iommu=soft" added in that > > > exposes the issue. > > > > Ok, so the problem really is the combination of pv-grub and iommu=soft. > > > > Any update on this problem? Thanks for reminding me. I fixed one problem (where the 'vfb' would hang pv-grub) but have't addressed this one. Here is what I get: Bootstrapping... Xen Minimal OS! start_info: 0xb9d000(VA) nr_pages: 0x40000 shared_inf: 0xb45ff000(MA) pt_base: 0xba0000(VA) nr_pt_frames: 0xb mfn_list: 0x99d000(VA) mod_start: 0x0(VA) mod_len: 0 flags: 0x0 cmd_line: (hd0,0)/grub/menu.lst stack: 0x95c980-0x97c980 MM: Init _text: 0x0(VA) _etext: 0x6bf94(VA) _erodata: 0x86000(VA) _edata: 0x8eae0(VA) stack start: 0x95c980(VA) _end: 0x99cf88(VA) start_pfn: bae max_pfn: 40000 Mapping memory range 0x1000000 - 0x40000000 setting 0x0-0x86000 readonly skipped 0x1000 MM: Initialise page allocator for da6000(da6000)-40000000(40000000) MM: done Demand map pfns at 40001000-2040001000. Heap resides at 2040002000-4040002000. Initialising timer interface Initialising console ... done. gnttab_table mapped at 0x40001000. Initialising scheduler Thread "Idle": pointer: 0x2040002050, stack: 0xfc0000 Initialising xenbus Thread "xenstore": pointer: 0x2040002800, stack: 0xfd0000 Dummy main: start_info=0x97ca80 Thread "main": pointer: 0x2040002fb0, stack: 0xfe0000 Thread "pcifront": pointer: 0x2040003760, stack: 0xff0000 pcifront_watches: waiting for backend path to appear device/pci/0/backend vbd 51712 is hd0 ******************* BLKFRONT for device/vbd/51712 ********** backend at /local/domain/0/backend/vbd/6/51712 [ 9955.307884] blkback: ring-ref 2047, event-channel 4, protocol 1 (x86_64-abi) Failed to read /local/domain/0/backend/vbd/6/51712/feature-flush-cache. 41943040 sectors of 512 bytes ************************** Thread "kbdfront": pointer: 0x2040004590, stack: 0x1000000 ******************* FBFRONT for device/vfb/0 ********** ******************* KBDFRONT for device/vkbd/0 ********** backend at /local/domain/0/backend/vkbd/6/0 /local/domain/0/backend/vkbd/6/0 connected ************************** KBDFRONT Thread "kbdfront" exited. backend at /local/domain/0/backend/vfb/6/0 /local/domain/0/backend/vfb/6/0 connected ************************** FBFRONT [ 9966.656265] vif6.0: no IPv6 routers present [ 9971.301098] switch: port 2(vif6.0) entering forwarding state Thread "kbdfront close": pointer: 0x2040004590, stack: 0x1000000 close fb: backend at /local/domain/0/backend/vfb/6/0 close kbd: backend at /local/domain/0/backend/vkbd/6/0 Thread "kbdfront close" exited. close blk: backend=/local/domain/0/backend/vbd/6/51712 node=device/vbd/51712 [ 9974.405828] frontend_changed: backend/vbd/6/51712: prepare for reconnect go! mapping kernel into physical memory about to get started... [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Linux version 2.6.34NEB (root@xxxxxxxxxxxxxxxxx) (gcc version 4.4.4 20100503 (Red Hat 4.4.4-2) (GCC) ) #2 SMP Fri Jun 25 15:01:02 EDT 2010 [ 0.000000] Command line: ro root=/dev/mapper/vg_vs03-lv_root rd_LVM_LV=vg_vs03/lv_root rd_LVM_LV=vg_vs03/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYTABLE=us debug earlyprintk=xen console=hvc0 iommu=swiotlb swiotlb=force [ 0.000000] ACPI in unprivileged domain disabled [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable) [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) [ 0.000000] Xen: 0000000000100000 - 0000000040000000 (usable) [ 0.000000] bootconsole [xenboot0] enabled [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] DMI not present or invalid. [ 0.000000] e820 update range: 0000000000000000 - 0000000000001000 (usable) ==> (reserved) [ 0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable) [ 0.000000] No AGP bridge found [ 0.000000] last_pfn = 0x40000 max_arch_pfn = 0x400000000 [ 0.000000] e820 update range: 0000000000001000 - 0000000000010000 (usable) ==> (reserved) [ 0.000000] Scanning 1 areas for low memory corruption [ 0.000000] modified physical RAM map: [ 0.000000] modified: 0000000000000000 - 0000000000010000 (reserved) [ 0.000000] modified: 0000000000010000 - 00000000000a0000 (usable) [ 0.000000] modified: 00000000000a0000 - 0000000000100000 (reserved) [ 0.000000] modified: 0000000000100000 - 0000000040000000 (usable) [ 0.000000] initial memory mapped : 0 - 20000000 [ 0.000000] init_memory_mapping: 0000000000000000-0000000040000000 [ 0.000000] 0000000000 - 0040000000 page 4k [ 0.000000] kernel direct mapping tables up to 40000000 @ 100000-302000 [ 0.000000] RAMDISK: 01a23000 - 024c0000 [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at 0000000000000000-0000000040000000 [ 0.000000] Initmem setup node 0 0000000000000000-0000000040000000 [ 0.000000] NODE_DATA [0000000001a02180 - 0000000001a0717f] [ 0.000000] Zone PFN ranges: [ 0.000000] DMA 0x00000010 -> 0x00001000 [ 0.000000] DMA32 0x00001000 -> 0x00100000 [ 0.000000] Normal empty [ 0.000000] Movable zone start PFN for each node [ 0.000000] early_node_map[2] active PFN ranges [ 0.000000] 0: 0x00000010 -> 0x000000a0 [ 0.000000] 0: 0x00000100 -> 0x00040000 [ 0.000000] On node 0 totalpages: 262032 [ 0.000000] DMA zone: 56 pages used for memmap [ 0.000000] DMA zone: 0 pages reserved [ 0.000000] DMA zone: 3928 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 3528 pages used for memmap [ 0.000000] DMA32 zone: 254520 pages, LIFO batch:31 [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] No local APIC present [ 0.000000] APIC: disable apic facility [ 0.000000] APIC: switched to apic NOOP [ 0.000000] nr_irqs_gsi: 16 [ 0.000000] early_res array is doubled to 64 at [16000 - 167ff] [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 [ 0.000000] Allocating PCI resources starting at 40000000 (gap: 40000000:c0000000) [ 0.000000] Booting paravirtualized kernel on Xen [ 0.000000] Xen version: 4.1-100622 (preserve-AD) [ 0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff8800026da000 s84776 r8192 d21720 u114688 [ 0.000000] pcpu-alloc: s84776 r8192 d21720 u114688 alloc=28*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [ 0.000000] trying to map vcpu_info 0 at ffff8800026e5020, mfn 1425c9, offset 32 [ 0.000000] cpu 0 using vcpu_info at ffff8800026e5020 [ 0.000000] trying to map vcpu_info 1 at ffff880002701020, mfn 1425ad, offset 32 [ 0.000000] cpu 1 using vcpu_info at ffff880002701020 [ 0.000000] trying to map vcpu_info 2 at ffff88000271d020, mfn 142591, offset 32 [ 0.000000] cpu 2 using vcpu_info at ffff88000271d020 [ 0.000000] trying to map vcpu_info 3 at ffff880002739020, mfn 142575, offset 32 [ 0.000000] cpu 3 using vcpu_info at ffff880002739020 [ 0.000000] Xen: using vcpu_info placement [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 258448 [ 0.000000] Policy zone: DMA32 [ 0.000000] Kernel command line: ro root=/dev/mapper/vg_vs03-lv_root rd_LVM_LV=vg_vs03/lv_root rd_LVM_LV=vg_vs03/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYTABLE=us debug earlyprintk=xen console=hvc0 iommu=swiotlb swiotlb=force [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) (XEN) mm.c:4137:d6 Bad page 00000000001331a9: ed=ffff83033ffc0000(6), sd=ffff83033ffc0000, caf=8000000000000002, taf=7400000000000001 (XEN) mm.c:4137:d6 Bad page 00000000001331a9: ed=ffff83033ffc0000(6), sd=ffff83033ffc0000, caf=8000000000000002, taf=7400000000000001 (XEN) mm.c:4137:d6 Bad page 00000000001331a9: ed=ffff83033ffc0000(6), sd=ffff83033ffc0000, caf=8000000000000002, taf=7400000000000001 (XEN) mm.c:4137:d6 Bad page 00000000001331a9: ed=ffff83033ffc0000(6), sd=ffff83033ffc0000, caf=8000000000000002, taf=7400000000000001 (XEN) mm.c:4137:d6 Bad page 00000000001331a9: ed=ffff83033ffc0000(6), sd=ffff83033ffc0000, caf=8000000000000002, taf=7400000000000001 (XEN) mm.c:4137:d6 Bad page 00000000001331a9: ed=ffff83033ffc0000(6), sd=ffff83033ffc0000, caf=8000000000000002, taf=7400000000000001 [ 0.000000] Kernel panic - not syncing: DMA(-12): Failed to exchange pages allocated for DMA with Xen! We either don\047t have the permission or you do not have enoughfree memory under 4GB! [ 0.000000] [ 0.000000] Pid: 0, comm: swapper Not tainted 2.6.34NEB #2 [ 0.000000] Call Trace: [ 0.000000] [<ffffffff8145373c>] panic+0x73/0xee [ 0.000000] [<ffffffff810052c2>] ? xen_create_contiguous_region+0xd3/0xee [ 0.000000] [<ffffffff818c6946>] xen_swiotlb_init+0x10a/0x142 [ 0.000000] [<ffffffff8145cf35>] ? _etext+0x0/0x3 [ 0.000000] [<ffffffff818a5763>] pci_xen_swiotlb_init+0x17/0x24 [ 0.000000] [<ffffffff818a808f>] pci_iommu_alloc+0x2f/0x36 [ 0.000000] [<ffffffff818b5c05>] mem_init+0x19/0xec [ 0.000000] [<ffffffff818a1b7c>] start_kernel+0x1ce/0x3ae [ 0.000000] [<ffffffff818a12c8>] x86_64_start_reservations+0xb3/0xb7 [ 0.000000] [<ffffffff818a437b>] xen_start_kernel+0x54b/0x54f [ 0.000000] BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 [ 0.000000] IP: [<ffffffff8122b7c0>] xen_evtchn_do_upcall+0xe5/0x19a [ 0.000000] PGD 0 [ 0.000000] Oops: 0000 [#1] SMP [ 0.000000] last sysfs file: [ 0.000000] CPU 0 [ 0.000000] Modules linked in: [ 0.000000] [ 0.000000] Pid: 0, comm: swapper Not tainted 2.6.34NEB #2 / [ 0.000000] RIP: e030:[<ffffffff8122b7c0>] [<ffffffff8122b7c0>] xen_evtchn_do_upcall+0xe5/0x19a [ 0.000000] RSP: e02b:ffff8800026ddf28 EFLAGS: 00010002 [ 0.000000] RAX: 0000000000000000 RBX: 00000000000114e0 RCX: 0000000000000002 [ 0.000000] RDX: 0000000000000100 RSI: 0000000000000000 RDI: 0000000000000000 [ 0.000000] RBP: ffff8800026ddfa8 R08: 0000000000000000 R09: ffffffff81801e38 [ 0.000000] R10: 0000000000000006 R11: 0000000000000000 R12: ffffffffffffffff [ 0.000000] R13: 0000000000000000 R14: ffffffffff57a000 R15: 00000000000114e0 [ 0.000000] FS: 0000000000000000(0000) GS:ffff8800026da000(0000) knlGS:0000000000000000 [ 0.000000] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 0.000000] CR2: 0000000000000008 CR3: 0000000001811000 CR4: 0000000000002660 [ 0.000000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 0.000000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 0.000000] Process swapper (pid: 0, threadinfo ffffffff81800000, task ffffffff81819020) [ 0.000000] Stack: [ 0.000000] 0000000000000000 0000000000000000 0000000000000140 0000000000000000 [ 0.000000] <0> ffff8800026e5028 0000000000000000 0000000000000000 0000000000000100 [ 0.000000] <0> ffff8800026e5020 0000000000000000 0000000000000000 0000000000000001 [ 0.000000] Call Trace: [ 0.000000] <IRQ> [ 0.000000] [<ffffffff8100a92e>] xen_do_hypervisor_callback+0x1e/0x30 [ 0.000000] <EOI> [ 0.000000] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1006 [ 0.000000] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1006 [ 0.000000] [<ffffffff81041f74>] ? release_console_sem+0x18f/0x1c7 [ 0.000000] [<ffffffff8100647d>] ? xen_force_evtchn_callback+0xd/0xf [ 0.000000] [<ffffffff81006b52>] ? check_events+0x12/0x20 [ 0.000000] [<ffffffff81006af9>] ? xen_irq_enable_direct_end+0x0/0x7 [ 0.000000] [<ffffffff814537b0>] ? panic+0xe7/0xee [ 0.000000] [<ffffffff810052c2>] ? xen_create_contiguous_region+0xd3/0xee [ 0.000000] [<ffffffff818c6946>] ? xen_swiotlb_init+0x10a/0x142 [ 0.000000] [<ffffffff8145cf35>] ? _etext+0x0/0x3 [ 0.000000] [<ffffffff818a5763>] ? pci_xen_swiotlb_init+0x17/0x24 [ 0.000000] [<ffffffff818a808f>] ? pci_iommu_alloc+0x2f/0x36 [ 0.000000] [<ffffffff818b5c05>] ? mem_init+0x19/0xec [ 0.000000] [<ffffffff818a1b7c>] ? start_kernel+0x1ce/0x3ae [ 0.000000] [<ffffffff818a12c8>] ? x86_64_start_reservations+0xb3/0xb7 [ 0.000000] [<ffffffff818a437b>] ? xen_start_kernel+0x54b/0x54f [ 0.000000] Code: 00 00 00 89 55 ac 48 8d 90 40 01 00 00 48 05 00 01 00 00 48 89 4d 98 48 89 45 b8 48 89 55 90 eb 2c 48 0f bc c9 03 4d ac 48 63 c9 <8b> 14 88 83 fa ff 74 f8 89 d7 89 55 88 e8 59 92 e5 ff 48 85 c0 [ 0.000000] RIP [<ffffffff8122b7c0>] xen_evtchn_do_upcall+0xe5/0x19a [ 0.000000] RSP <ffff8800026ddf28> [ 0.000000] CR2: 0000000000000008 [ 0.000000] ---[ end trace 4eaa2a86a8e2da22 ]--- [ 0.000000] Kernel panic - not syncing: Fatal exception in interrupt [ 0.000000] Pid: 0, comm: swapper Tainted: G D 2.6.34NEB #2 [ 0.000000] Call Trace: [ 0.000000] <IRQ> [<ffffffff8145373c>] panic+0x73/0xee [ 0.000000] [<ffffffff814565c0>] oops_end+0xb7/0xc7 [ 0.000000] [<ffffffff8102d91b>] no_context+0x1f7/0x206 [ 0.000000] [<ffffffff8102daa9>] __bad_area_nosemaphore+0x17f/0x1a2 [ 0.000000] [<ffffffff8102dada>] bad_area_nosemaphore+0xe/0x10 [ 0.000000] [<ffffffff81458617>] do_page_fault+0x18b/0x2a3 [ 0.000000] [<ffffffff81455a75>] page_fault+0x25/0x30 [ 0.000000] [<ffffffff8122b7c0>] ? xen_evtchn_do_upcall+0xe5/0x19a [ 0.000000] [<ffffffff8122b745>] ? xen_evtchn_do_upcall+0x6a/0x19a [ 0.000000] [<ffffffff8100a92e>] xen_do_hypervisor_callback+0x1e/0x30 [ 0.000000] <EOI> [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1006 [ 0.000000] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1006 [ 0.000000] [<ffffffff81041f74>] ? release_console_sem+0x18f/0x1c7 [ 0.000000] [<ffffffff8100647d>] ? xen_force_evtchn_callback+0xd/0xf [ 0.000000] [<ffffffff81006b52>] ? check_events+0x12/0x20 [ 0.000000] [<ffffffff81006af9>] ? xen_irq_enable_direct_end+0x0/0x7 [ 0.000000] [<ffffffff814537b0>] ? panic+0xe7/0xee [ 0.000000] [<ffffffff810052c2>] ? xen_create_contiguous_region+0xd3/0xee [ 0.000000] [<ffffffff818c6946>] ? xen_swiotlb_init+0x10a/0x142 [ 0.000000] [<ffffffff8145cf35>] ? _etext+0x0/0x3 [ 0.000000] [<ffffffff818a5763>] ? pci_xen_swiotlb_init+0x17/0x24 [ 0.000000] [<ffffffff818a808f>] ? pci_iommu_alloc+0x2f/0x36 [ 0.000000] [<ffffffff818b5c05>] ? mem_init+0x19/0xec [ 0.000000] [<ffffffff818a1b7c>] ? start_kernel+0x1ce/0x3ae [ 0.000000] [<ffffffff818a12c8>] ? x86_64_start_reservations+0xb3/0xb7 [ 0.000000] [<ffffffff818a437b>] ? xen_start_kernel+0x54b/0x54f I wonder if this is a similar problem that another user of Xen has reported when they use blktap with pv-grub - it puts some weird flags on the I/O pages so during shutdown the guest is in a weird state.. _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-devel

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.