[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [PATCH v5 0/9] toolstack-based approach to pvhvm guest kexec

This patch series provides x86 PVHVM domains with an ability to perform

Changes from v4:
- "on_soft_reset" option was introduced, now it's possible to specify the 
  [Wei Liu]
- renamed libxl__domain_soft_reset_destroy_old to 
- libxl__domain_soft_reset_destroy now takes gc instead of ctx, coding style fix
  [Wei Liu]
- remove forgotten 'd_config.b_info.nodemap.size = 0' hackaround, add
  reload_domain_config() on soft reset path [Wei Liu]
- add whole procedure description to libxl_internal.h [Wei Liu]
- reword 'recipient' description in struct domain [Julien Grall]

Nothing was done with regards to ARM. It seems that correct implementation of
mfn_to_gmfn() returning propper gmfn for DomUs is the desirable solution in the
long run, however, any additional thoughts here are more than wellcome! 

Changes from RFCv3:
This is the first non-RFC series as no major concerns were expressed. I'm trying
to address Jan's comments. Changes are:
- Move from XEN_DOMCTL_set_recipient to XEN_DOMCTL_devour (I don't really like
  the name but nothing more appropriate came to my mind) which incorporates
  former XEN_DOMCTL_set_recipient and XEN_DOMCTL_destroydomain to prevent
  original domain from changing its allocations during transfer procedure.
- Check in free_domheap_pages() that assign_pages() succeeded.
- Change printk() in free_domheap_pages().
- DOMDYING_locked state was introduced to support XEN_DOMCTL_devour.
- xc_domain_soft_reset() got simplified a bit. Now we just wait for the original
  domain to die or loose all its pages.
- rebased on top of current master branch.

Changes from RFC/WIPv2:

Here is a slightly different approach to memory reassignment. Instead of
introducing new (and very doubtful) XENMEM_transfer operation introduce
simple XEN_DOMCTL_set_recipient operation and do everything in 
handler utilizing normal domain destroy path. This is better because:
- The approach is general-enough
- All memory pages are usually being freed when the domain is destroyed
- No special grants handling required
- Better supportability

With regards to PV:
Though XEN_DOMCTL_set_recipient works for both PV and HVM this patchset does not
bring PV kexec/kdump support. xc_domain_soft_reset() is limited to work with HVM
domains only. The main reason for that is: it is (in theory) possible to save 
and rebuild them with the new domain but that would only allow us to resume 
from where we stopped. If we want to execute new kernel we need to build the 
kernel/initrd/bootstrap_pagetables/... structure we build to boot PV domain 
That however would destroy the original domain's memory thus making kdump 
To make everything work additional support from kexec userspace/linux kernel is
required and I'm not sure it makes sense to implement all this stuff in the 
light of

Original description:

When a PVHVM linux guest performs kexec there are lots of things which
require taking care of:
- shared info, vcpu_info
- grants
- event channels
- ...
Instead of taking care of all these things we can rebuild the domain
performing kexec from scratch doing so-called soft-reboot.

The idea was suggested by David Vrabel, Jan Beulich, and Konrad Rzeszutek Wilk.

Previous discussions:
This patch series:

on resetting VCPU_info (and that's where 'rebuild everything with the
toolstack solution' was suggested):
Previous versions:

EVTCHNOP_reset (got merged):

P.S. The patch series can be tested with PVHVM Linux guest with the following

diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index c0cb11f..33c5cdd 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -33,6 +33,10 @@
 #include <linux/memblock.h>
 #include <linux/edd.h>

+#include <linux/kexec.h>
 #include <xen/xen.h>
 #include <xen/events.h>
 #include <xen/interface/xen.h>
@@ -1810,6 +1814,22 @@ static struct notifier_block xen_hvm_cpu_notifier = {
   .notifier_call   = xen_hvm_cpu_notify,

+static void xen_pvhvm_kexec_shutdown(void)
+       native_machine_shutdown();
+       if (kexec_in_progress) {
+          xen_reboot(SHUTDOWN_soft_reset);
+          }
+static void xen_pvhvm_crash_shutdown(struct pt_regs *regs)
+       native_machine_crash_shutdown(regs);
+       xen_reboot(SHUTDOWN_soft_reset);
 static void __init xen_hvm_guest_init(void)
@@ -1826,6 +1846,10 @@ static void __init xen_hvm_guest_init(void)
   x86_init.irqs.intr_init = xen_init_IRQ;
+       machine_ops.shutdown = xen_pvhvm_kexec_shutdown;
+       machine_ops.crash_shutdown = xen_pvhvm_crash_shutdown;

 static bool xen_nopv = false;
diff --git a/include/xen/interface/sched.h b/include/xen/interface/sched.h
index 9ce0839..b5942a8 100644
--- a/include/xen/interface/sched.h
+++ b/include/xen/interface/sched.h
@@ -107,5 +107,6 @@ struct sched_watchdog {
 #define SHUTDOWN_suspend    2  /* Clean up, save suspend info, kill.         */
 #define SHUTDOWN_crash      3  /* Tell controller we've crashed.             */
 #define SHUTDOWN_watchdog   4  /* Restart because watchdog time expired.     */
+#define SHUTDOWN_soft_reset 5  /* Soft-reset for kexec.                      */

 #endif /* __XEN_PUBLIC_SCHED_H__ */

Vitaly Kuznetsov (9):
  xen: introduce DOMDYING_locked state
  xen: introduce SHUTDOWN_soft_reset shutdown reason
  libxl: support SHUTDOWN_soft_reset shutdown reason
  xen: introduce XEN_DOMCTL_devour
  libxc: support XEN_DOMCTL_devour
  libxl: add libxl__domain_soft_reset_destroy()
  libxc: introduce soft reset for HVM domains
  libxl: soft reset support
  xsm: add XEN_DOMCTL_devour support

 docs/man/xl.cfg.pod.5               |  12 ++
 tools/libxc/Makefile                |   1 +
 tools/libxc/include/xenctrl.h       |  14 ++
 tools/libxc/include/xenguest.h      |  20 +++
 tools/libxc/xc_domain.c             |  13 ++
 tools/libxc/xc_domain_soft_reset.c  | 282 ++++++++++++++++++++++++++++++++++++
 tools/libxl/libxl.c                 |  33 ++++-
 tools/libxl/libxl.h                 |   6 +
 tools/libxl/libxl_create.c          | 103 +++++++++++--
 tools/libxl/libxl_internal.h        |  30 ++++
 tools/libxl/libxl_types.idl         |   4 +
 tools/libxl/xl_cmdimpl.c            |  37 ++++-
 tools/python/xen/lowlevel/xl/xl.c   |   1 +
 xen/common/domain.c                 |   4 +
 xen/common/domctl.c                 |  39 +++++
 xen/common/page_alloc.c             |  28 +++-
 xen/common/shutdown.c               |   7 +
 xen/include/public/domctl.h         |  15 ++
 xen/include/public/sched.h          |   3 +-
 xen/include/xen/sched.h             |   5 +-
 xen/include/xsm/dummy.h             |   6 +
 xen/include/xsm/xsm.h               |   6 +
 xen/xsm/dummy.c                     |   1 +
 xen/xsm/flask/hooks.c               |  17 +++
 xen/xsm/flask/policy/access_vectors |  10 ++
 25 files changed, 674 insertions(+), 23 deletions(-)
 create mode 100644 tools/libxc/xc_domain_soft_reset.c


Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.