[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH v6 1/2] docs/designs: Add a design document for non-cooperative live migration

Hi Paul,

On 05/03/2020 17:30, pdurrant@xxxxxxxx wrote:
From: Paul Durrant <pdurrant@xxxxxxxxxx>

It has become apparent to some large cloud providers that the current
model of cooperative migration of guests under Xen is not usable as it
relies on software running inside the guest, which is likely beyond the
provider's control.
This patch introduces a proposal for non-cooperative live migration,
designed not to rely on any guest-side software.

Signed-off-by: Paul Durrant <pdurrant@xxxxxxxxxx>
Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
Cc: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
Cc: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
Cc: Jan Beulich <jbeulich@xxxxxxxx>
Cc: Julien Grall <julien@xxxxxxx>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>
Cc: Wei Liu <wl@xxxxxxx>

  - Addressed comments from Julien

  - Note that PV domain are not just expected to co-operate, they are
    required to

  - Fix issues raised by Wei

  - Use the term 'non-cooperative' instead of 'transparent'
  - Replace 'trust in' with 'reliance on' when referring to guest-side
  docs/designs/non-cooperative-migration.md | 280 ++++++++++++++++++++++
  1 file changed, 280 insertions(+)
  create mode 100644 docs/designs/non-cooperative-migration.md

diff --git a/docs/designs/non-cooperative-migration.md 
new file mode 100644
index 0000000000..e28161562f
--- /dev/null
+++ b/docs/designs/non-cooperative-migration.md
@@ -0,0 +1,280 @@
+# Non-Cooperative Migration of Guests on Xen
+## Background
+The normal model of migration in Xen is driven by the guest because it was
+originally implemented for PV guests, where the guest must be aware it is
+running under Xen and is hence expected to co-operate. This model dates from
+an era when it was assumed that the host administrator had control of at
+least the privileged software running in the guest (i.e. the guest kernel)
+which may still be true in an enterprise deployment but is not generally
+true in a cloud environment. The aim of this design is to provide a model
+which is purely host driven, requiring no co-operation from the software
+running in the guest, and is thus suitable for cloud scenarios.
+PV guests are out of scope for this project because, as is outlined above,
+they have a symbiotic relationship with the hypervisor and therefore a
+certain level of co-operation is required.
+HVM guests can already be migrated on Xen without guest co-operation but
+only if they don’t have PV drivers installed[1] or are not in power state
+S0. The reason for not expecting co-operation if the guest is any sort of

Sorry to be picky, but this still not clear this is x86/ACPI specific.

How about "x86 HVM guests can already be migrated on Xen without guest-co-operation but only if they don't have PV drivers installed[1] or are not in ACPI power state S0".


+For a PV guest the `arch_shared_info` sub-structure contains important
+information about the guest’s P2M, but this information is not relevant for
+an HVM guest where the P2M is not directly manipulated via the guest. The
+other state contained in the `shared_info` structure relates the domain
+wall-clock (the state of which should already be transferred by the `RTC`
+HVM context information which contained in the `HVM_CONTEXT` save record)
+and some event channel state (particularly if using the *2l* protocol).
+Event channel state will need to be fully transferred if we are not going
+to require the guest co-operation to re-open the channels and so it should
+be possible to re-build a shared info page for an HVM guest from such other
+Note that the shared info page also contains an array of
+`XEN_LEGACY_MAX_VCPUS` (32) `vcpu_info` structures. A domain may nominate

I would say (32 for x86)

+a different guest physical address to use for the vcpu info. Fix x86

Do you mean 'For' instead of 'Fix'.

+domains this is mandatory if a domain wants to use more than 32 vCPUs
+and optional otherwise. This mapping is not currently transferred in the
I would make this sentence arch-agnostic. e.g: "This is mandatory if a domain wants to use more than `XEN_LEGACY_MAX_VCPUS` and optional otherwise".

+migration state so this will either need to be added into an existing save
+record, or an additional type of save record will be needed.


Julien Grall

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.