Re: [Xen-devel] [PATCH v5 1/2] docs/designs: Add a design document for non-cooperative live migration

[Julien Grall <julien@xxxxxxx>]

Hi Paul,

On 04/03/2020 15:23, Durrant, Paul wrote:
> > -----Original Message-----
> > From: Xen-devel <xen-devel-bounces@xxxxxxxxxxxxxxxxxxxx> On Behalf Of Julien 
> > Grall
> > Sent: 04 March 2020 15:11
> > To: Durrant, Paul <pdurrant@xxxxxxxxxxxx>; xen-devel@xxxxxxxxxxxxxxxxxxxx
> > Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>; Wei Liu <wl@xxxxxxx>; Konrad 
> > Rzeszutek Wilk
> > <konrad.wilk@xxxxxxxxxx>; George Dunlap <George.Dunlap@xxxxxxxxxxxxx>; Andrew 
> > Cooper
> > <andrew.cooper3@xxxxxxxxxx>; Ian Jackson <ian.jackson@xxxxxxxxxxxxx>; Jan Beulich 
> > <jbeulich@xxxxxxxx>
> > Subject: Re: [Xen-devel] [PATCH v5 1/2] docs/designs: Add a design document for 
> > non-cooperative live
> > migration
> >
> > Hi Paul,
> >
> > The proposal looks sensible to me. Some NITpicking below.
> >
> > On 13/02/2020 10:53, Paul Durrant wrote:
> > > It has become apparent to some large cloud providers that the current
> > > model of cooperative migration of guests under Xen is not usable as it
> > > relies on software running inside the guest, which is likely beyond the
> > > provider's control.
> > > This patch introduces a proposal for non-cooperative live migration,
> > > designed not to rely on any guest-side software.
> > >
> > > Signed-off-by: Paul Durrant <pdurrant@xxxxxxxxxx>
> > > ---
> > > Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> > > Cc: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
> > > Cc: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
> > > Cc: Jan Beulich <jbeulich@xxxxxxxx>
> > > Cc: Julien Grall <julien@xxxxxxx>
> > > Cc: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
> > > Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>
> > > Cc: Wei Liu <wl@xxxxxxx>
> > >
> > > v5:
> > >    - Note that PV domain are not just expected to co-operate, they are
> > >      required to
> > >
> > > v4:
> > >    - Fix issues raised by Wei
> > >
> > > v2:
> > >    - Use the term 'non-cooperative' instead of 'transparent'
> > >    - Replace 'trust in' with 'reliance on' when referring to guest-side
> > >      software
> > > ---
> > >    docs/designs/non-cooperative-migration.md | 272 ++++++++++++++++++++++
> > >    1 file changed, 272 insertions(+)
> > >    create mode 100644 docs/designs/non-cooperative-migration.md
> > >
> > > diff --git a/docs/designs/non-cooperative-migration.md 
> > > b/docs/designs/non-cooperative-migration.md
> > > new file mode 100644
> > > index 0000000000..09f74c8c0d
> > > --- /dev/null
> > > +++ b/docs/designs/non-cooperative-migration.md
> > > @@ -0,0 +1,272 @@
> > > +# Non-Cooperative Migration of Guests on Xen
> > > +
> > > +## Background
> > > +
> > > +The normal model of migration in Xen is driven by the guest because it was
> > > +originally implemented for PV guests, where the guest must be aware it is
> > > +running under Xen and is hence expected to co-operate. This model dates from
> > > +an era when it was assumed that the host administrator had control of at least
> > > +the privileged software running in the guest (i.e. the guest kernel) which may
> > > +still be true in an enterprise deployment but is not generally true in a cloud
> > > +environment. The aim of this design is to provide a model which is purely host
> > > +driven, requiring no co-operation from the software running in the
> > > +guest, and is thus suitable for cloud scenarios.
> > > +
> > > +PV guests are out of scope for this project because, as is outlined above, they
> > > +have a symbiotic relationship with the hypervisor and therefore a certain level
> > > +of co-operation is required.
> > > +
> > > +HVM guests can already be migrated on Xen without guest co-operation but only
> > > +if they don’t have PV drivers installed[1] or are in power state S3. The
> > S3 is very ACPI centric, so I would prefer if we avoid the term. I think
> > the non-ACPI description is "suspend to RAM". I would be OK is you
> > mention S3 in parenthesis.
> I'm actually pulling this from the way the code is currently written, which is 
> clearly quite x86 specific:
>
> xc_hvm_param_get(CTX->xch, domid, HVM_PARAM_ACPI_S_STATE, &hvm_s_state)
> .
> .
> .
> if (dsps->type == LIBXL_DOMAIN_TYPE_HVM && (!hvm_pvdrv || hvm_s_state)) {
>      LOGD(DEBUG, domid, "Calling xc_domain_shutdown on HVM domain");
>      ret = xc_domain_shutdown(CTX->xch, domid, SHUTDOWN_suspend);
>      .
>      .
> }
>
> So actually I should say 'not in power state S0'.
I understand that the current code is x86 specific. Arm would likely 
have a similar requirement although not based on ACPI.
However, my point here is nothing in the document says it is focusing on 
x86 only. The concept itself is not arch specific, the document is 
mostly x86 free except in a couple of bits. So I would like them to be 
rewritten in an arch-agnostic way.
Note that I am ok with arch-specific example.

Cheers,

--
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel