Re: [Xen-devel] [PATCH v3 00/10] xen/arm: live migration support in arndale board

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Thu, 2013-08-01 at 21:57 +0900, Jaeyong Yoo wrote:
> Hi all,
> here goes the v3 patch series for live migration in arndale board.

Was a v4 of this ever posted (with the vlpt stuff)? If so I appear to
have missed it, sorry.

Feature freeze for Xen 4.4 is currently 18th October[0], we'd really
like to see live migration support before then! (which in reality means
it needs to be posted well before to give ample time for review of what
is likely to be a complex patch set)

Ian.

[0] http://article.gmane.org/gmane.comp.emulators.xen.devel/168132 


> This version applies the comments from v2 patch series, which are majorly:
> 
>   1) just use one timer struct for storing vtimer and ptimer
>      for hvm context: patch 1
> 
>   2) for dirty page tracing, use virtual-linear page table for accessing
>      guest p2m in xen: patch 6, 7, and 9
> 
>   3) Rather than using hard-coded guest memory map in xen, use the one from
>      toolstack by implementing set/get_memory_map hypercall: patch 3 and 10
> 
> This patch series does not support SMP guest migration. We are expecting
> to provide SMP-guests live migration in version 4 patch series.
> 
> We also have tested the stability of v3 patch series as follows:
> 
>   - setup two arndale boards with xen (let's say A and B)
>   - launch 3 domUs at A
>   - simulataneously migrate 3 domUs from A to B back and forth.
>   - we say one round of migration if all 3 domUs migrate
>     from A to B and migrate back from B to A.
> 
> When we perform the above tests without any load on domUs, the migration
> goes to 80~100 rounds. After that, dom0 suddenly stops responding. When we
> perform with network load (iperf on each domU), the migration goes to 2~3
> rounds and dom0 stops responding.
> 
> After several repeated tests, we gather the PCs where dom0 stucks, and those 
> are
>   - _raw_spin_lock: (called by try_to_wake_up)
>   - panic_smp_self_stop
>   - cpu_v7_dcache_clean_area
> 
> I think those bugs are somehow related to live migration or maybe other parts
> that may result in complicated cause-and-effect chain to live migration. In 
> any
> case, I would like to look into the detail to figure out the cause and 
> possibly
> fix the bugs.
> 
> In the meanwhile, I would appreciate your comments to this patch series :)
> 
> 
> Best,
> Jaeyong
> 
> lexey Sokolov, Elena Pyatunina, Evgeny Fedotov, and Nikolay Martyanov (1):
>   xen/arm: Implement toolstack for xl restore/save and migrate
> 
> Alexey Sokolov (1):
>   xen/arm: Implement modify_returncode
> 
> Evgeny Fedotov (2):
>   xen/arm: Implement set_memory_map hypercall
>   xen/arm: Implement get_maximum_gpfn hypercall for arm
> 
> Jaeyong Yoo and Evgeny Fedotov (1):
>   xen/arm: Implement hvm save and restore
> 
> Jaeyong Yoo and Alexey Sokolov (1):
>   xen/arm: Add more registers for saving and restoring vcpu  registers
> 
> Jaeyong Yoo and Elena Pyatunina (2)
>   xen/arm: Add handling write fault for dirty-page tracing
>   xen/arm: Implement hypercall for dirty page tracing (shadow op)
> 
> Jaeyong Yoo (2):
>   xen/arm: Implement virtual-linear page table for guest      p2m
>     mapping in live migration
>   xen/arm: Fixing clear_guest_offset macro
> 
>  config/arm32.mk                          |   1 +
>  tools/include/xen-foreign/reference.size |   2 +-
>  tools/libxc/Makefile                     |   5 +
>  tools/libxc/xc_arm_migrate.c             | 686 
> +++++++++++++++++++++++++++++++
>  tools/libxc/xc_dom_arm.c                 |  12 +-
>  tools/libxc/xc_domain.c                  |  44 ++
>  tools/libxc/xc_resume.c                  |  25 ++
>  tools/libxc/xenctrl.h                    |  23 ++
>  tools/misc/Makefile                      |   4 +
>  xen/arch/arm/Makefile                    |   2 +
>  xen/arch/arm/domain.c                    |  44 ++
>  xen/arch/arm/domctl.c                    | 137 +++++-
>  xen/arch/arm/hvm.c                       | 124 ++++++
>  xen/arch/arm/mm.c                        | 284 ++++++++++++-
>  xen/arch/arm/p2m.c                       | 307 ++++++++++++++
>  xen/arch/arm/save.c                      |  66 +++
>  xen/arch/arm/setup.c                     |   3 +
>  xen/arch/arm/traps.c                     |  16 +-
>  xen/arch/arm/vlpt.c                      | 162 ++++++++
>  xen/common/Makefile                      |   2 +
>  xen/include/asm-arm/config.h             |   3 +
>  xen/include/asm-arm/domain.h             |  13 +
>  xen/include/asm-arm/guest_access.h       |   5 +-
>  xen/include/asm-arm/hvm/support.h        |  29 ++
>  xen/include/asm-arm/mm.h                 |   7 +
>  xen/include/asm-arm/p2m.h                |   4 +
>  xen/include/asm-arm/processor.h          |   2 +
>  xen/include/asm-arm/vlpt.h               |  10 +
>  xen/include/public/arch-arm.h            |  35 ++
>  xen/include/public/arch-arm/hvm/save.h   |  41 ++
>  xen/include/public/memory.h              |  15 +-
>  xen/include/xsm/dummy.h                  |   5 +
>  xen/include/xsm/xsm.h                    |   5 +
>  33 files changed, 2113 insertions(+), 10 deletions(-)
>  create mode 100644 tools/libxc/xc_arm_migrate.c
>  create mode 100644 xen/arch/arm/save.c
>  create mode 100644 xen/arch/arm/vlpt.c
>  create mode 100644 xen/include/asm-arm/hvm/support.h
>  create mode 100644 xen/include/asm-arm/vlpt.h
> 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel