Re: [Xen-devel] RE: Ballooning up

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Mon, 2010-09-13 at 22:17 +0100, Dan Magenheimer wrote:
> > From: Jeremy Fitzhardinge [mailto:jeremy@xxxxxxxx]
> > Cc: Dan Magenheimer; Daniel Kiper; Stefano Stabellini; Konrad Rzeszutek
> > Wilk
> > Subject: Ballooning up
> > 
> >  I finally got around to implementing "ballooning up" in the pvops
> > kernels.  Now if you start a domain with "memory=X maxmem=Y", the
> > domain
> > will start with X MB of memory, but you can use "x[ml] mem-set" to
> > expand the domain up to Y.
> 
> Nice!
> 
> > As a side-effect, it also works for dom0.  If you set dom0_mem on the
> > Xen command line, then nr_pages is limited to that value, but the
> > kernel
> > can still see the system's real E820 map, and therefore adds all the
> > system's memory to its own balloon driver, potentially allowing dom0 to
> > expand up to take all physical memory.
> > 
> > However, this may caused bad side-effects if your system memory is much
> > larger than your dom0_mem, especially if you use a 32-bit dom0.  I may
> > need to add a kernel command line option to limit the max initial
> > balloon size to mitigate this...
> 
> I would call this dom0 functionality a bug.  I think both Citrix
> and Oracle use dom0_mem as a normal boot option for every
> installation and, while I think both employ heuristics to choose
> a larger dom0_mem for larger physical memory, I don't think it
> grows large enough for, say, >256GB physical memory, to accommodate
> the necessarily large number of page tables.

FWIW XenServer statically uses dom0_mem=752M and then balloons down on
smaller systems where so much domain 0 memory is not required, the
minimum is 128M or 256M or something.

A 32on64 domain 0 kernel fails to boot if dom0_mem is more than around
~56G because it runs out of lowmem for the page array. I suspect that
for some period before that the system isn't terribly usable due to low
amounts of available lowmem, even if it does manage to boot.

> So, I'd vote for NOT allowing dom0 to balloon up to physical
> memory if dom0_mem is specified, and possibly a kernel command
> line option that allows it to grow beyond.  Or, possibly, no
> option and never allow dom0 memory to grow beyond dom0_mem
> unless (possibly) it grows with hot-plug.
> 
> Dan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel