Re: [Xen-devel] [PATCH] Boot PV guests with more than 128GB (v2) for 3.7

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 01.08.12 at 17:50, Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx> wrote:
> With these patches I've gotten it to boot up to 384GB. Around that area
> something weird happens - mainly the pagetables that the toolstack allocated
> seems to have missing data. I hadn't looked in details, but this is what
> domain builder tells me:
> 
> 
> xc_dom_alloc_segment:   ramdisk      : 0xffffffff82278000 -> 
> 0xffffffff930b4000  (pfn 0x2278 + 0x10e3c pages)
> xc_dom_malloc            : 1621 kB
> xc_dom_pfn_to_ptr: domU mapping: pfn 0x2278+0x10e3c at 0x7fb0853a2000
> xc_dom_do_gunzip: unzip ok, 0x4ba831c -> 0x10e3be10
> xc_dom_alloc_segment:   phys2mach    : 0xffffffff930b4000 -> 
> 0xffffffffc30b4000  (pfn 0x130b4 + 0x30000 pages)
> xc_dom_malloc            : 4608 kB
> xc_dom_pfn_to_ptr: domU mapping: pfn 0x130b4+0x30000 at 0x7fb0553a2000
> xc_dom_alloc_page   :   start info   : 0xffffffffc30b4000 (pfn 0x430b4)
> xc_dom_alloc_page   :   xenstore     : 0xffffffffc30b5000 (pfn 0x430b5)
> xc_dom_alloc_page   :   console      : 0xffffffffc30b6000 (pfn 0x430b6)
> nr_page_tables: 0x0000ffffffffffff/48: 0xffff000000000000 -> 
> 0xffffffffffffffff, 1 table(s)
> nr_page_tables: 0x0000007fffffffff/39: 0xffffff8000000000 -> 
> 0xffffffffffffffff, 1 table(s)
> nr_page_tables: 0x000000003fffffff/30: 0xffffffff80000000 -> 
> 0xffffffffffffffff, 2 table(s)
> nr_page_tables: 0x00000000001fffff/21: 0xffffffff80000000 -> 
> 0xffffffffc33fffff, 538 table(s)
> xc_dom_alloc_segment:   page tables  : 0xffffffffc30b7000 -> 
> 0xffffffffc32d5000  (pfn 0x430b7 + 0x21e pages)
> xc_dom_pfn_to_ptr: domU mapping: pfn 0x430b7+0x21e at 0x7fb055184000
> xc_dom_alloc_page   :   boot stack   : 0xffffffffc32d5000 (pfn 0x432d5)
> xc_dom_build_image  : virt_alloc_end : 0xffffffffc32d6000
> xc_dom_build_image  : virt_pgtab_end : 0xffffffffc3400000
> 
> Note it is is 0xffffffffc30b4000 - so already past the level2_kernel_pgt 
> (L3[510]
> and in level2_fixmap_pgt territory (L3[511]).
> 
> At that stage we are still operating using the Xen provided pagetable - which
> look to have the L4[511][511] empty! Which sounds to me like a Xen tool-stack
> problem? Jan, have you seen something similar to this?

No we haven't, but I also don't think anyone tried to create as
big a DomU. I was, however, under the impression that DomU-s
this big had been created at Oracle before. Or was that only up
to 256Gb perhaps?

In any case, setup_pgtables_x86_64() indeed looks flawed
to me: While the clearing of l1tab looks right, l[23]tab get
cleared (and hence a new table allocated) too early. l2tab
should really get cleared only when l1tab gets cleared _and_
the L2 clearing condition is true. Similarly for l3tab then, and
of course - even though it would unlikely ever matter -
setup_pgtables_x86_32_pae() is broken in the same way.

Afaict this got broken with the domain build re-write between
3.0.4 and 3.1 (the old code looks alright).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel