[Xen-devel] [RFC] Nested Paging Live Migration

["Huang2, Wei" <Wei.Huang2@xxxxxxx>]

The attached file supports AMD-V nested paging live migration. Please comment. I will create an updated version after collecting feedbacks.

 

 arch/x86/hvm/hvm.c            |    2
 arch/x86/hvm/io.c             |    2
 arch/x86/hvm/svm/svm.c        |    3
 arch/x86/mm.c                 |   12 +-
 arch/x86/mm/hap/hap.c         |  220 +++++++++++++++++++++++++++++++++++++++++-
 arch/x86/mm/p2m.c             |   92 +++++++++++++++--
 arch/x86/mm/paging.c          |   12 ++
 include/asm-x86/domain.h      |    8 +
 include/asm-x86/grant_table.h |    2
 include/asm-x86/hap.h         |    1
 include/asm-x86/p2m.h         |    5
 include/asm-x86/page.h        |    2
 include/asm-x86/paging.h      |    2
 include/asm-x86/shadow.h      |    7 -
 14 files changed, 341 insertions(+), 29 deletions(-)

Design:

1. We handle four live migration operators as follow:
* XEN_DOMCTL_SHADOW_OP_ENABLE_LOGDIRTY
** Allocates log_dirty_bitmap
** Set log dirty bit in paging mode
** Goes through the P2M table and mark all physical memory as NOT WRITABLE
** Continues to run the guest as usual

 

* XEN_DOMCTL_SHADOW_OP_PEEK
** There is nothing special here. It is pretty similar to shadow code. Just copy dirty bitmap information to live migration handler.

 

* XEN_DOMCTL_SHADOW_OP_CLEAN
** Clean dirty bitmap to all 0's.
** Goes through the P2M table and marks all physical memory as NOT WRITABLE
** Continues to run the guest as usual

 

* XEN_DOMCTL_SHADOW_OP_OFF
** Fix P2M table and mark all physical memory as WRITABLE
** De-allocate dirty bitmap resources
** Clear log dirty bit in paging mode

 

2. We handle nested page fault as follow:

* Nested Paging Fault
** If it is MMIO space, call handle_mmio()
** Otherwise, call p2m_fix_table() to mark a specific page as WRITABLE. Additionally, we call paging_mark_dirty() to update dirty bitmap. By doing this, we only receive one NPF for each dirty page (in each cycle).

 

The following areas require special attention:

1. paging_mark_dirty()
Currently, paging_mark_dirty() dispatches to sh_mark_dirty() or hap_mark_dirty() based on paging support. I personally prefer a function pointer. However, current paging interface only provides a function pointer for vcpu-level functions, not for domain-level functions. This is a bit annoying.

 

2. locking in p2m_set_l1e_flags()
p2m_set_l1e_flags(), which is invoked by hap.c, calls hap_write_p2m_entry(). hap_lock() is called twice. I currently remove hap_lock() in hap_write_p2m_entry(). A better solution is needed here.

 

 

Thanks,

 

-Wei

Attachment: npt_live_migrate_RFC.txt
Description: npt_live_migrate_RFC.txt

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel