RE: [Xen-devel] xen: memory initialization/balloon fixes (#3)

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> From: David Vrabel [mailto:david.vrabel@xxxxxxxxxx]
> Subject: Re: [Xen-devel] xen: memory initialization/balloon fixes (#3)
> 
> On 23/09/11 00:46, Dan Magenheimer wrote:
> >> From: Jeremy Fitzhardinge [mailto:jeremy@xxxxxxxx]
> >>
> >> I don't think the Xen parts allocate/reserves lots of memory
> >> unnecessarily, so it shouldn't be too different from the 2.6.18-xen
> >> kernels.  They do reserve various chunks of memory, but for things like
> >> RAMDISK I think they get released again (and anyway, I don't think
> >> that's going to be anywhere near 30MB, let alone 60).  I'm not very
> >> confident in those /proc/meminfo numbers - they may count memory as
> >> "reserved" if its in a reserved region even if the pages themselves have
> >> been released to the kernel pool.
> >
> > No, the first line of /proc/meminfo is precisely "totalram_pages".
> 
> I think most of the increase in reserved memory compared to classic Xen
> kernels is the change to using the generic SWIOTLB.  This is up to 64 MiB.

Hi David --

My data agrees with Konrad's reply.  I don't see the SWIOTLB
but am only testing with smaller guests (<2GB).

> >>>>> Again, this looks like the correct behavior to me.
> >>>> Hmmm... so if a user (or automated tool) uses the Xen-defined
> >>>> API (i.e. /sys/devices/system/xen_memory/xen_memory0/target_kb)
> >>>> to use the Xen balloon driver to attempt to reduce memory usage
> >>>> to 100MB, and the Xen balloon driver instead reduces it to
> >>>> some random number somewhere between 40MB and 90MB, which
> >>>> may or may not cause OOMs, you consider this correct behavior?
> >>> I still think this is a bug but apparently orthogonal to
> >>> your patchset.  So sorry to bother you.
> >>
> >> If you ask for 100MB, it should never try to make the domain smaller
> >> than that; if it does, it suggests the number is being misparsed or
> >> something.

(Jeremy -- No it's not getting mis-parsed... in the existing balloon
code the try-to-balloon-to-target-N code is instead ballooning to
N-D, where D is "reserved_pages+absent_pages"... and as you pointed
out, this sum gets modified after the balloon is initialized.)

> > OK then balloon_stats.current_pages can never be larger than totalram_pages.
> > Which means that balloon_stats.current_pages must always grow
> > and shrink when totalram_pages does (which is true now only in
> > the balloon driver code).  Which means, I think:
> >
> > balloon_stats.current_pages is just plain wrong!  It doesn't need to
> > exist!  If we replace every instance in balloon.c with totalram_pages,
> > I think everything just works.  Will run some tests tomorrow.
> 
> No.  balloon_stats.current_pages is the amount of pages used by the
> domain from Xen's point of view (and must be equal to the amount report
> by xl top).  It is not what the guest kernel thinks is the number of
> usable pages.

OK.  It still appears to be not always accurate to me.  Are you using
balloon_stats.current_pages via sysfs?  AFAICT, the interface used to
get the amount of domain memory (e.g. for xl top) does not use
balloon_stats.current_pages, so the only way it would be visible
outside of the balloon driver is via sysfs.  I suppose that is
part of the guest ABI now, even if it contains an unreliable value.

In any case, I can leave balloon_stats.current_pages alone in
my patch-under-development.  It is "just plain wrong" for my
purposes and for setting a balloon target, and I think still wrong
for the purpose you state, but I think I can leave unmodified the
code that updates it so as not to affect code that uses it via
sysfs.

> Because totalram_pages doesn't include some reserved pages
> balloon_stats.current_pages will necessarily always be greater.

Yep.

> If you're attempting to make the domain self-balloon I don't see why
> you're even interested in the total number of pages.  Surely it's the
> number of free pages that's useful?

No, after the kernel has been busy awhile, the number of free
pages can be quite small, because most of RAM gets used in the
page cache.  Self-ballooning (as used with tmem) is much more
aggressive than that because part of the purpose of tmem is
to act as an all-domain page cache.  If you're not familiar with
tmem, see http://oss.oracle.com/projects/tmem.

Thanks again for the feedback.  I'll cc you on my upcoming patch.

Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel