[Xen-devel] RE: tmem - really default to on?

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

Sorry for the delay... I was away from the office.

> >Rather than turn it off in Xen, maybe best solution is to turn it
> >off in non-pvops dom0, at least by default.  Or really there's
> >no reason it needs to be on for dom0 at all, so Xen code could
> >check for dom0 and disable it if dom0.
> 
> I don't know that much about tmem, but why would Dom0 not benefit
> from it being enabled if there's memory available in Xen?

Oops, yes, I misspoke.  There are uses for tmem from dom0,
especially when other guests are using "file:" virtual disks
(i.e. *not* O_DIRECT) and those other guests are not tmem-aware.

> >And the tmem code that limits to single page allocation should IIRC
> >only be invoked once all system memory has been absorbed (which
> >I suppose could be the case for dom0 when dom0_mem is not specified?)
> 
> Or if enough VMs had been created to consume all freely available
> (outside of tmem) memory.
>
> As said in the original mail - we have seen systems with more than 2G
> "freeable" memory still causing domain creation to fail with 
> "not enough memory".

I can guess at a few ways this could happen, but would probably
need more info about the specific environment to be sure:

A) There is some order>0 memory allocation in Xen domain creation
that doesn't fall back to order=0, that I've not seen in my testing
but shows up in your systems (or has very recently been added).
If true, this is a problem not only for tmem but also for all other
memory optimization work and we need to identify and (if possible)
fix it.

B) The code in domain creation that frees freeable memory (see
tools/python/xen/xend/balloon.py, search for "tmem") is computing
a too-small value, maybe due to recent changes reserving "hidden
extra" memory for domains).

C) The above code is not getting executed at all during domain
creation, perhaps because you have different/old domain creation
tools that aren't aware of tmem (or use some API that
I didn't know about and consequently didn't test).

Any thoughts?

Thanks,
Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel