[Xen-devel] Re: xen 3.1.2: can't unpin bootstrap page

[Manuel Bouyer <bouyer@xxxxxxxxxxxxxxx>]

On Mon, Dec 03, 2007 at 07:12:43PM +0100, Manuel Bouyer wrote:
> [ I'm not subscribed to the list, please keep me in Cc: ]
> Hi,
> I'm trying to upgrade the NetBSD xen packages to xen-3.1.2 and I'm having
> troubles loading a x86_64 NetBSD kernel. The hypervisor crashes very
> early in the bootstrap setup, when trying to unping its bootstrap page table
> after switching to the new page table. Basically NetBSD does the following:
> build temporary new pages tables mapping the kernel in the free memory area
> after the Xen page tables, pin and switch to the new temporary L4 page,
> unpin the old boot L4 page provided by Xen. After that it should map the
> boostrap pages provided by Xen R/W and build the final pages tables
> for the kernel in place of the Xen-provided bootstrap pages and switch to
> it, but it doesn't get there: the hypervisor panics with:
> (XEN) Assertion '(x & ((1U<<16)-1)) != 0' failed at mm.c:1670
> 
> I added a MEM_LOG() in put_page_from_l4e and here's what I get (there
> is also some debug infos from the NetBSD kernel, especially about the
> temporary pages tables being built):
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to 
> Xen).
> (XEN) Freed 116kB init memory.
> xen_arch_pmap_bootstrap init_tables=0xffffffff80caa000
> xen_bootstrap_tables(0xffffffff80caa000, 0xffffffff80d16000, 11, 9)
> xen_bootstrap_tables text_end 0xffffffff80a10000 map_end 0xffffffff80d28000
> L3 va 0xffffffff80d17000 pa 0xd17000 entry 0x3ad17007 -> L4[0x1ff]
> L2 va 0xffffffff80d18000 pa 0xd18000 entry 0x3ad18007 -> L3[0x1fe]
> L1 va 0xffffffff80d19000 pa 0xd19000 entry 0x3ad19007 -> L2[0x0]
> L1 va 0xffffffff80d1a000 pa 0xd1a000 entry 0x3ad1a007 -> L2[0x1]
> L1 va 0xffffffff80d1b000 pa 0xd1b000 entry 0x3ad1b007 -> L2[0x2]
> L1 va 0xffffffff80d1c000 pa 0xd1c000 entry 0x3ad1c007 -> L2[0x3]
> L1 va 0xffffffff80d1d000 pa 0xd1d000 entry 0x3ad1d007 -> L2[0x4]
> L1 va 0xffffffff80d1e000 pa 0xd1e000 entry 0x3ad1e007 -> L2[0x5]
> va 0xffffffff80caa000 pa 0xcaa000 entry 0x3acaa005 -> L1[0xaa]
> L1 va 0xffffffff80d1f000 pa 0xd1f000 entry 0x3ad1f007 -> L2[0x6]
> L1 va 0xffffffff80d20000 pa 0xd20000 entry 0x3ad20007 -> L2[0x7]
> L1 va 0xffffffff80d21000 pa 0xd21000 entry 0x3ad21007 -> L2[0x8]
> bt_pgd[PDIR_SLOT_PTE] va 0xffffffff80d16000 pa 0xd16000 entry 0x3ad16005
> pin PDG
> switch to PDG
> bt_pgd[PDIR_SLOT_PTE] now entry 0x3ad16005
> L4_BASE va 0x7fbfdfeff000
> value 0x0
> [PDIR_SLOT_PTE] 0x3ad16025
> unpin old PDG
> (XEN) mm.c:881:d0 put_page_from_l4e entry 0x3e3bf063 pfn 0x3acaa type 0x0

I tracked it down to an entry in the L4 page provided by Xen:
slot 0 has 0x3e3bf063 in it, which isn't even in the domains's memory range.
Trying to clear this entry from the dom0 kernel leads to the same hypervisor
panic.

Here's what Xen says about this dom0:
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0xffffffff80100000 memsz=0x810328
(XEN) elf_parse_binary: phdr: paddr=0xffffffff80a10340 memsz=0xedf98
(XEN) elf_parse_binary: memory: 0xffffffff80100000 -> 0xffffffff80afe2d8  
(XEN) elf_xen_parse: __xen_guest: "GUEST_OS=NetBSD,GUEST_VER=4.99,XEN_VER=xen-3.
0,LOADER=generic,VIRT_BASE=0xffffffff80000000,ELF_PADDR_OFFSET=0xffffffff8000000
0,VIRT_ENTRY=0xffffffff80100000,HYPERCALL_PAGE=0x00000101,BSD_SYMTAB=yes" 
(XEN) elf_xen_parse_guest_info: GUEST_OS="NetBSD"
(XEN) elf_xen_parse_guest_info: GUEST_VER="4.99"
(XEN) elf_xen_parse_guest_info: XEN_VER="xen-3.0"
(XEN) elf_xen_parse_guest_info: LOADER="generic"
(XEN) elf_xen_parse_guest_info: VIRT_BASE="0xffffffff80000000"
(XEN) elf_xen_parse_guest_info: ELF_PADDR_OFFSET="0xffffffff80000000"
(XEN) elf_xen_parse_guest_info: VIRT_ENTRY="0xffffffff80100000"
(XEN) elf_xen_parse_guest_info: HYPERCALL_PAGE="0x00000101"
(XEN) elf_xen_parse_guest_info: BSD_SYMTAB="yes"
(XEN) elf_xen_addr_calc_check: addresses:
(XEN)     virt_base        = 0xffffffff80000000
(XEN)     elf_paddr_offset = 0xffffffff80000000
(XEN)     virt_offset      = 0x0
(XEN)     virt_kstart      = 0xffffffff80100000
(XEN)     virt_kend        = 0xffffffff80be8c70
(XEN)     virt_entry       = 0xffffffff80100000
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80100000 -> 0xffffffff80afe2d8
(XEN)  Dom0 symbol map 0xffffffff80afe2d8 -> 0xffffffff80be8c70
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   000000003a000000->000000003b000000 (94208 pages to be 
allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff80100000->ffffffff80be8c70
(XEN)  Init. ramdisk: ffffffff80be9000->ffffffff80be9000
(XEN)  Phys-Mach map: ffffffff80be9000->ffffffff80ca9000
(XEN)  Start info:    ffffffff80ca9000->ffffffff80ca949c
(XEN)  Page tables:   ffffffff80caa000->ffffffff80cb5000
(XEN)  Boot stack:    ffffffff80cb5000->ffffffff80cb6000
(XEN)  TOTAL:         ffffffff80000000->ffffffff81000000
(XEN)  ENTRY ADDRESS: ffffffff80100000
(XEN) Dom0 has maximum 2 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xffffffff80100000 -> 0xffffffff80910328 
(XEN) elf_load_binary: phdr 1 at 0xffffffff80a10340 -> 0xffffffff80a7dae0 
(XEN) elf_load_bsdsyms: shdr 22 at 0xffff83003d1153a4 -> 0xffffffff80afe960
(XEN) elf_load_bsdsyms: shdr 23 at 0xffff83003d115b38 -> 0xffffffff80afeab8
(XEN) elf_load_bsdsyms: shdr 24 at 0xffff83003d1a2e68 -> 0xffffffff80b8bde8
(XEN) Scrubbing Free RAM: .....done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to 
Xen).
(XEN) Freed 116kB init memory.

-- 
Manuel Bouyer, LIP6, Universite Paris VI.           Manuel.Bouyer@xxxxxxx
     NetBSD: 26 ans d'experience feront toujours la difference
--

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel