Re: [Xen-devel] can not use all available memory

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> From: Sander Eikelenboom [mailto:linux@xxxxxxxxxxxxxx]
> Subject: Re: [Xen-devel] can not use all available memory
> 
> Monday, November 26, 2012, 5:58:28 PM, you wrote:
> 
> >> From: Ian Campbell [mailto:Ian.Campbell@xxxxxxxxxx]
> >> Sent: Friday, November 23, 2012 6:34 AM
> >> To: Pasi KÃrkkÃinen
> >> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; Alexander Bienzeisler
> >> Subject: Re: [Xen-devel] can not use all available memory
> >>
> >> On Fri, 2012-11-23 at 13:29 +0000, Pasi KÃrkkÃinen wrote:
> >> > On Fri, Nov 23, 2012 at 01:15:53PM +0000, Ian Campbell wrote:
> >> > > On Fri, 2012-11-23 at 12:55 +0000, Pasi KÃrkkÃinen wrote:
> >> > >
> >> > > > > This has been discussed at length on the list before, please check 
> >> > > > > the
> >> > > > > archives.
> >> > > > >
> >> > > >
> >> > > > I tried googling quickly but I didn't find anything relevant..
> >> > >
> >> > > I found
> >> > > http://web.archiveorange.com/archive/v/zCKz5T3PLvtyZDSPQc9i
> >> > > in a matter of seconds, then:
> >> > > http://lists.xen.org/archives/html/xen-devel/2011-11/msg01415.html
> >> > > http://lists.xen.org/archives/html/xen-users/2012-05/msg00139.html
> >> > > http://lists.xen.org/archives/html/xen-users/2012-05/msg00146.html
> >> > >
> >> > > > To me this behaviour still seems wrong. What's the point of 
> >> > > > autoballoon=1 trying to
> >> > > > balloon down dom0 if the hypervisor already has enough free memory 
> >> > > > for the VM ?
> >> > > >
> >> > > > In this case:
> >> > > >         - dom0_mem=2G
> >> > > >         - new VM to launch with size 16 GB.
> >> > > >         - Xen has 28 GB of free memory.
> >> > > >
> >> > > > So clearly there's no need to try to balloon down dom0..
> >> > >
> >> > > Right, so don't set autoballoon then.
> >> > >
> >> >
> >> > It's enabled as a default.. so many people hit this problem.
> >> >
> >> >
> >> > > > not-yet-implemented check/feature in xl, or a bug?
> >> > >
> >> > > Neither, it is the intended behaviour of xl autoballoon, this option
> >> > > means exactly "take the required memory from dom0".
> >> > >
> >> >
> >> > http://xenbits.xen.org/docs/4.2-testing/man/xl.conf.5.html
> >> >
> >> > "autoballoon=BOOLEAN
> >> >
> >> >     If disabled then xl will not attempt to reduce the amount of
> >> > memory assigned to domain 0 in order to create free memory when
> >> > starting a new domain. You should set this if you use the dom0_mem
> >> > hypervisor command line to reduce the amount of memory given to domain
> >> > 0 by default.
> >> >
> >> >     Default: 1"
> >> >
> >> >
> >> > I think we should modify that to say "You should set autoballoon=0 if 
> >> > you use the dom0_mem
> >> hypervisor command line .."
> >> > At least I understood that text in the opposite way..
> >>
> >> Yes, there should be a s/set/unset/ in there I think.
> >>
> >> > Also: What happens if you have autoballoon=1 and you start some VMs, 
> >> > then stop the VMs,
> >> > so most of the memory is now free in Xen..
> >>
> >> xl balloons dom0 back up when it destroy domains with autoballoon=1.
> >>
> >> > and then you try to start a big VM ?
> >> > Aren't you going to hit the same problem as in this thread?
> 
> > Hmmm... this behavior and default may make sense for the Citrix
> > memory model (single machine, dom0 is "the user" so you want it
> > to always hold most of physical RAM not used by guests).  But
> > probably not so for a more cloud-like memory model.
> 
> > Is there any (easy) way to force autoballoon=0 if the hypervisor
> > dom0_mem boot option is specified?  Or is there some reasonably
> > sane case I am missing where a user would want both dom0_mem
> > and autoballoon=1?
> 
> > Oracle VM always boots servers with dom0_mem= set so (if/when
> > OVM switches to use xl), OVM will always set autoballoon off.
> > So it's the large number of non-Citrix-non-Oracle Xen-as-a-service
> > providers I am trying to help here.
> 
> 
> 
> Hmm i was bitten by this 2 weeks ago, i found it a bit not intuitive that:
> - While the guest i was trying to start required less memory than was freely 
> available (according to
> xentop) outside of dom0
> - It would fail, because xl started to try to balloon dom0 down which failed.

Hi Sander --

I could be wrong (and I am confident someone will correct me if I am) but
I think this is because the Citrix memory model assumes there is an
inference-driven policy engine for load-balancing memory across competing
virtual machines ("squeezed").  I suspect squeezed returns unallocated
xen "free" memory to dom0.

IMHO, such policy engines are good for demos and so salespeople can
say "yes, this product supports memory overcommit" but Transcendent
Memory goes quite a bit further (albeit not for guests with proprietary
OS's).

Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel