[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Xen balloon driver improvement (version 1)



Hi all

This is my initial design to improve Xen balloon driver.

PDF version with graphs can be found at

http://xenbits.xen.org/people/liuw/xen-balloon-driver-improvement.pdf

% Xen Balloon Driver Improvement
% Wei Liu <<wei.liu2@xxxxxxxxxx>>

-------------------------------------------
Version     Date         Changes
-------     ----         ------------------
  1         22/10/2014   Initial version.
-------------------------------------------

## Motives

1. Balloon pages fragments guest physical address space.
1. Balloon compaction infrastructure can migrate ballooned pages from
   start of zone to end of zone, hence creating contiguous guest physical
   address space.
1. Having contiguous guest physical address enables some options to
   improve performance.

## Goal of improvement

Balloon driver makes use of as many huge pages as possible,
defragmenting both guest address space and Xen pages. This should be
achieved without any particular hypervisor side feature.

## Design and implementation

When balloon driver is asked to increase / decrease reservation, it
will always start with huge page. However, due to resource
availability in both hypervisor and guest, it's not always possible to
get hold of a huge page. In that case the driver will fall back to use
normal size page. Balloon driver later will try to coalesce small size
pages into huge page. As time goes by, both Xen and guest should use
more and more huge pages.

To achieve the said goal, several changes will be made:

1. Make use of balloon page compaction.
1. Maintain multiple queues for pages of different sizes and purposes.
1. Periodically exchange normal size pages with huge pages.

### Make use of balloon page compaction

Balloon page migration moves balloon pages from start of zone to end
of zone, making guest physical address space contiguous. This gives
worker thread to allocate huge pages in order to coalesce small pages.

Currently, Xen balloon driver gets its page directly from page
allocator. To enable balloon page migration, those pages now need to
be allocated from core balloon driver. Pages allocated from core
balloon driver are subject to balloon page compaction.

The use of balloon compaction doesn't require introducign new
interfaces between Xen balloon driver and the rest of the system. Most
changes are internal to Xen balloon driver.

Xen balloon driver will also need to provide a callback to migrate
balloon page. In essence callback function receives "old page", which
is a already ballooned out page, and "new page", which is a page to be
ballooned out, then it inflates "old page" and deflates "new page".

The core of migration callback is XENMEM\_exchange hypercall. This
makes sure that inflation of old page and deflation of new page is
done atomically, so even if a domain is beyond its memory target and
being enforced, it can still compact memory.

### Maintain multiple queues for pages of different sizes and purposes

We maintain multiple queues for pages of different sizes inside Xen
balloon driver, so that Xen balloon worker thread can coalesce smaller
size pages into one larger size page. Queues for special purposed
pages, such as balloon pages used to map foreign pages, are also
maintained. These special purposed pages are not subject to migration
and page coalescence.

For instance, balloon driver can maintain three queues:

1. queue for 2 MB pages
1. queue for 4 KB pages (delegated to core balloon driver)
1. queue for pages used to mapped pages from other domain

More queues can be added when necessary, but for now one queue for
normal pages and one queue for huge page should be enough.

### Worker thread to coalesce small size pages

Worker thread wakes up periodically to check if there's enough pages
in normal size page queue to coalesce into a huge page. If so, it will
try to exchange that huge page into a number of normal size pages with
XENMEM\_exchange hypercall.

## Flowcharts

These flowcharts assume normal page size is 4K and huge page size is
2M.  They show how two queues are maintained.

![Increase Reservation](increase-reservation.png)

![Decrease Reservation](decrease-reservation.png)

![Exchange Pages](exchange-pages.png)


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.