Re: [Xen-devel] Xen balloon driver improvement (version 1)

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Wed, Oct 22, 2014 at 06:32:57PM +0100, Andrew Cooper wrote:
> On 22/10/14 17:29, Wei Liu wrote:
> > Hi all
> >
> > This is my initial design to improve Xen balloon driver.
> >
> > PDF version with graphs can be found at
> >
> > http://xenbits.xen.org/people/liuw/xen-balloon-driver-improvement.pdf
> 
> +1 pandoc  (makes far nicer pdfs)
> 
> I have a few style recommendations, and some grammar issues
> 

I will fix them.

[...]
> > 1. Balloon compaction infrastructure can migrate ballooned pages from
> >    start of zone to end of zone, hence creating contiguous guest physical
> >    address space.
> 
> What does start of zone and end of zone mean?  While this document is

It's Linux concept. Zone is just ... memory zone. :-)

> labelled "Xen balloon driver improvements", it applies mainly to the
> Linux balloon driver, and should be noted as such.
> 

Sure.

> > 1. Having contiguous guest physical address enables some options to
> >    improve performance.
> >
> > ## Goal of improvement
> >
[...]
> > Balloon page migration moves balloon pages from start of zone to end
> > of zone, making guest physical address space contiguous. This gives
> > worker thread to allocate huge pages in order to coalesce small pages.
> 
> I can't parse this final sentence.
> 

This gives worker thread *a chance* to allocate huge pages in order to
coalesce small pages.

> >
> > Currently, Xen balloon driver gets its page directly from page
[...]
> > ### Maintain multiple queues for pages of different sizes and purposes
> >
> > We maintain multiple queues for pages of different sizes inside Xen
> > balloon driver, so that Xen balloon worker thread can coalesce smaller
> > size pages into one larger size page. Queues for special purposed
> > pages, such as balloon pages used to map foreign pages, are also
> > maintained. These special purposed pages are not subject to migration
> > and page coalescence.
> >
> > For instance, balloon driver can maintain three queues:
> >
> > 1. queue for 2 MB pages
> > 1. queue for 4 KB pages (delegated to core balloon driver)
> > 1. queue for pages used to mapped pages from other domain
> 
> What about 1GB pages?
> 

I wouldn't bother with 1GB pages here.

It would require too much work to coalesce 4KB pages to 1GB pages. 

And probably due to resource limitation, 1GB allocation is likely to
fail. If you have a guest with hundreds of GB ram you wouldn't bother
using balloon compaction in the first place.

So having a 1GB queue seems too much effort with too little gain.

> >
> > More queues can be added when necessary, but for now one queue for
> > normal pages and one queue for huge page should be enough.
> >
> > ### Worker thread to coalesce small size pages
> >
> > Worker thread wakes up periodically to check if there's enough pages
> 
> "there are enough pages"
> 
> > in normal size page queue to coalesce into a huge page. If so, it will
> > try to exchange that huge page into a number of normal size pages with
> > XENMEM\_exchange hypercall.
> 
> Your diagram says "exchange 2M and 4K pages".  Exchange how, because you
> cannot exchange a set of scattered 4K pages for a 2M contiguous one in
> an HVM guest.
> 

2M page is populated, while 4K pages are not.

So for the exchange hypercall, 2M is exchange.in and the list of 4K
pages is the output. In the end, 2M page becomes unpopulated, 4K pages
are populated.

I read the implementation of XENMEM_exchange, it looks OK for
me to do so. Did I overlook something?

Wei.

> ~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel