Re: [Xen-devel] domain creation vs querying free memory (xend and xl)

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> From: Ian Jackson [mailto:Ian.Jackson@xxxxxxxxxxxxx]
> Sent: Friday, September 28, 2012 11:12 AM
> To: Dan Magenheimer
> Cc: xen-devel@xxxxxxxxxxxxx; Kurt Hackel; Konrad Wilk
> Subject: Re: [Xen-devel] domain creation vs querying free memory (xend and xl)
> 
> Dan Magenheimer writes ("[Xen-devel] domain creation vs querying free memory 
> (xend and xl)"):
> > But the second domain launch fails, possibly after
> > several minutes because, actually, there isn't enough
> > physical RAM for both.
> 
> This is a real problem.  The solution is not easy, and may not make it
> for 4.3.  It would involve a rework of the memory handling code in
> libxl.

[broadening cc to "Xen memory technology people", please forward/add
if I missed someone]

Hi Ian --

If you can estimate the difficulty, it would appear you have
a specific libxl design in mind?  Maybe it would be useful to
brainstorm a bit to see if there might be a simpler/different
solution?

Bearing in mind that I know almost nothing about xl or
the tools layer, and that, as a result, I tend to look
for hypervisor solutions, I'm thinking it's not possible to
solve this without direct participation of the hypervisor anyway,
at least while ensuring the solution will successfully
work with any memory technology that involves ballooning
with the possibility of overcommit (i.e. tmem, page sharing
and host-swapping, manual ballooning, PoD)...  EVEN if the
toolset is single threaded (i.e. only one domain may
be created at a time, such as xapi). [1]

As a result, I've cc'ed other parties involved in memory
technologies who can chime in if they think the above
statement is incorrect for their technology...

Back to design brainstorming:

The way I am thinking about it, the tools need to be involved
to the extent that they would need to communicate to the
hypervisor the following facts (probably via new hypercall):

X1) I am launching a domain X and it is eventually going to
   consume up to a maximum of N MB.  Please tell me if
   there is sufficient RAM available AND, if so, reserve
   it until I tell you I am done. ("AND" implies transactional
   semantics)
X2) The launch of X is complete and I will not be requesting
   the allocation of any more RAM for it.  Please release
   the reservation, whether or not I've requested a total
   of N MB.

The calls may be nested or partially ordered, i.e.
   X1...Y1...Y2...X2
   X1...Y1...X2...Y2
and the hypervisor must be able to deal with this.

Then there would need to be two "versions" of "xm/xl free".
We can quibble about which should be the default, but
they would be:

- "xl --reserved free" asks the hypervisor how much RAM
   is available taking into account reservations
- "xm --raw free" asks the hypervisor for the instantaneous
   amount of RAM unallocated, not counting reservations

When the tools are not launching a domain (that is there
has been a matching X2 for all X1), the results of the
above "free" queries are always identical.

So, IanJ, does this match up with the design you were thinking
about?

Thanks,
Dan

[1] I think the core culprits are (a) the hypervisor accounts for
memory allocation of pages strictly on a first-come-first-served
basis and (b) the tools don't have any form of need-this-much-memory
"transaction" model

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel