Re: [Xen-devel] RFC: QEMU bumping memory limit and domain restore

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Tue, Jun 02, 2015 at 06:40:56PM +0100, Wei Liu wrote:
> On Tue, Jun 02, 2015 at 05:11:02PM +0100, Andrew Cooper wrote:
> > On 02/06/15 16:49, Ian Campbell wrote:
> > > On Tue, 2015-06-02 at 15:08 +0100, Wei Liu wrote:
> > > [...]
> > >>> So here is a proof of concept patch to record and honour that value
> > >>> during migration.  A new field is added in IDL. Note that we don't
> > >>> provide xl level config option for it and mandate it to be default value
> > >>> during domain creation. This is to prevent libxl user from using it to
> > >>> avoid unforeseen repercussions.
> > >> [...]
> > >>> This field is mandated to be default value during guest creation to
> > >>> avoid unforeseen repercussions. It's only honour when restoring a guest.
> > > IMHO this means that the libxl API/IDL is the wrong place for this
> > > value. Only user and/or application serviceable parts belong in the API.
> > >
> > > So while I agree that this value need to be communicated across a
> > > migration, the JSON blob is not the right mechanism for doing so. IOW if
> > > you go down this general path I think you need a new
> > > field/record/whatever in the migration protocol at some layer or other
> > > (if not libxc then at the libxl layer).
> > >
> > > To my mind this "actual state" vs "user configured state" is more akin
> > > to the sorts of things which is in the hypervisor save blob or something
> > > like that (nb: This is not a suggestion that it should go there).
> > >
> > > IIRC Don also outlined another case, which is
> > >     xl create -p
> > >     xl migrate
> > >     xl unpause
> > >
> > > Which might need more thought if any bumping can happen after the
> > > migrate i.e. on unpause?
> > >
> > >
> > 
> > The problem is qemu using set_max_mem.  It should never have done so.
> > 
> > Nothing other than libxl should be using such hypercalls, at which point
> > libxls idea of guest memory is accurate and the bug ceases to exist.
> > 
> 
> That would be an ideal solution, but it's not going to materialise
> any time soon because:
> 
> 1. Libxl cannot know how much more memory guest needs prior to QEMU
>    start-up.
> 2. QEMU cannot communicate back the value to libxl because
>  2.1 there is no communication channel;
>  2.2 there is no libxld to update the config (maybe we can work around
>      this by waiting for QEMU during guest creation).
> 
> While we work towards that end, a short term solution might be:
> 
> 1. Make QEMU not call xc_domain_setmaxmem.
> 2. Provide a field in IDL and a config option in xl to indicate how
>    much more memory is needed. User can fill in that option as he / she
>    sees fit.
> 
> After we get to the ideal solution we can then deprecate that field /
> option.
> 

An alternative we came up with during our IRL discussion.

1. QEMU writes the size of additional memory in xenstore.
2. Libxl record that in toolstack save path.
3. Remote end calls xc_domain_setmaxmem in toolstack restore path.

Wei.

> Wei.
> 
> > ~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel