Re: [Xen-devel] [PATCH] RFC: initial libxl support for xenpaging

[George Dunlap <george.dunlap@xxxxxxxxxx>]

On Tue, 2012-02-21 at 09:59 +0000, Ian Campbell wrote:
> On Mon, 2012-02-20 at 14:48 +0000, George Dunlap wrote:
> > On Mon, Feb 20, 2012 at 11:12 AM, Olaf Hering <olaf@xxxxxxxxx> wrote:
> > > On Mon, Feb 20, George Dunlap wrote:
> > >
> > >> Start-of-day:
> > >>  xenpaging off: Set balloon target to M, use PoD
> > >>  xenpaging on: ??
> > >>  xenpaging delay: Set balloon target to M, use PoD.  Wait
> > >> $pagingdelayboot seconds, if target not reached, set paging?
> > >
> > > Is the delay required?
> > > If paging and PoD target is M, xenpaging will do nothing because the
> > > guest can not exceed M (it will crash with OOM).
> > 
> > Ah, of course -- you don't need paging because it already has M
> > memory.  Forgot about that.
> > 
> > It would be nice, of course, if the pager could act as a back-fill for
> > these PoD pages; but that's another project, I think.
> > 
> > So that leaves us with:
> > 
> > maxmem=X
> > memory=M
> > xenpaging=[off|on|delay]
> > pagingdelay=60
> 
> FWIW these two can be expressed as:
> xenpaging=[off|on]
> pagingdelay=[0|60]
> 
> (and lets drop the "xen" prefix)
> 
> [...]
> > xl mem-set domain M
> >  xenpaging off: Set balloon target to M
> >  xenpaging on: Set paging target to M
> >  xenpaging delay: Set balloon target to M, and wait for actual memory
> > to reach M.  If it hasn't reached it by $paging_delay seconds, set
> > balloon target to M.
> 
> Did you mean "paging target" the second time you said "balloon target"
> in this one? I'll assume so. 

Er, yes, that's what I meant. :-)

> I would also suggest 
>       s/If it hasn't reached it by/After/
> since I think that will simplify things somewhat and setting page target
> to M makes no odds if the guest has ballooned to M.
> 
> I don't really like mem-set having such completely different behaviour
> depending on whether paging is on or off.
> 
> As you described before having paging on == set paging and balloon
> target to M results in fairly suboptimal behaviour and the name would
> also lead to people thinking it is the one they should use.
> 
> So why not make the "on" case the same as your "delay" case and do away
> with the distinction? If advanced users really want what you describe as
> "on" then they can set the delay to 0.

The only thing with this is that then the command will by default pause
until we reach the target, which may be several seconds.  We should make
sure to print a message saying that's what we're doing, so users don't
get confused.

> If the paging daemon could be start/stopped on demand (rather than being
> a domain build time choice) we could even consider making paging the
> default.
> 
> > xl mem-balloon-set domain M
> >  Set balloon target to M
> > xl mem-paging-set domain M
> >  Set paging target to M
> 
> How do these interact with mem-set. Especially in the delay case?

My idea was that "xl mem-paging-set domain M" would basically be
equivalent to xenstore-write /local/[whatever]/[whatever]: that is, no
attempt at synchronization.  If I do "xl mem-set" with a delay in one
window, then do "xl mem-paging-set" in another window, the first one
will take effect immediately, then the other one will take effect after
the delay.  If that's not what you want, either Ctrl-C the first one or
wait for it to finish. :-)

> e.g. would mem-paging-set disable the after delay behaviour of mem-set?
> Should we have "mem-paging-set domain auto" to turn that back on?

That's one of the reasons I conceived having mem-balloon-set: if you
want to switch to full-manual, you just switch to using full-manual
commands.  If/when you have stuff in a state that the simpler command
will work again, you can switch back.

> We also need to consider the behaviour of mem-set to increase things.
> Obviously you don't want to leave paging target set to the smaller value
> for a minute after setting the balloon target. I think we want to set it
> straight away in that case, if not before setting the balloon.

Yes; if we're increasing the target, we should set paging-target
immediately.

> How about the following? I've tried to include the "user facing"
> description as well as the actual implementation. I think the "user
> facing" portion is actually where we disagree but I also suspect that we
> may not actually disagree -- it's just that we are talking in terms of
> implementation so we don't see that the user facing interface is the
> same in what we are each thinking of ;-)
> 
> maxmem=X                      # maximum RAM the domain can ever see
> memory=M                      # current amount of RAM seen by the domain
> paging=[off|on]                       # allow the amount of memory a guest 
>                               # thinks it has to differ from the
>                               # amount actually available to it (its
>                               # "footprint")
> pagingauto=[off|on] (dflt=on)   # enable automatic enforcement of 
>                               # "footprint" for guests which do not
>                               # voluntarily obey changes to memory=M 
> pagingdelay=60                        # amount of time to give a guest to 
>                               # voluntarily comply before enforcing a 
>                               # footprint
> 
> xl mem-set domain M
>         Sets the amount of RAM which the guest believes it has available
>         to M. The guest should arrange to use only that much RAM and
>         return the rest to the hypervisor (e.g. by using a balloon
>         driver). If the guest does not do so then the host may use
>         technical means to enforce the guest's footprint of M. The guest
>         may suffer a performance penalty for this enforcement.
> 
>       paging off:     set balloon target to M.
>       paging on:      set balloon target to M.
>                       if pagingauto:
>                               wait delay IFF new target < old
>                               set paging target to M
>                               support -t <delay> to override default?
> 
> xl mem-paging-set domain N
>         Overrides the amount of RAM which the guest actually has
>         available (its "footprint") to N. The host will use technical
>         means to continue to provide the illusion to the guest that it
>         has memory=M (as adjusted by mem-set). There may be a
>         performance penalty for this.
>         
>       paging off:     error
>       paging on:      set paging target
>                       set pagingauto=off
> 
> xl mem-paging-set domain auto
>         Automatically manage paging. Request that the guest uses
>         memory=M (current value of memory, as adjusted by mem-set)
>         enforced when the guest is uncooperative (as described in
>         "mem-set")
>         
>       paging off:     error
>       paging on:      set paging target to M
>                       set pagingauto=on
> 
> No need for a separate balloon-set since that == mem-set with
> pagingauto=off.
> 
> Perhaps a separate "mem-paging-set domain manual" would be handy to
> enable that mode without having to remember M so you can use it as N 

I'd be OK with this.

> We could consider making "mem-paging-set domain N" fail with an error
> unless you previously set manual, to prevent users accidentally
> disabling the recommended automatic behaviour e.g. by typing
> mem-paging-set when they mean mem-set.
> 
> I liked Andres' suggestions of footprint as a term here BTW so I would
> prefer "mem-footprint-set" to "mem-paging-set" (at least I think so, I'm
> not 100% on that). If we don't have balloon-set then avoiding the name
> paging seems like a good idea too. Other possible names might be
> "mem-override-set" or something.

Well for one, "footprint" to me would imply "I don't care how you got
there, just make it take this much memory".  So saying in the docs that
"xl mem-set" would attempt to set the memory footprint would be OK.  But
I definitely don't think we should use "footprint" to mean "paging
target".  Even apart from the fact that the name to me means something
else, the config options are called "paging".  In any case, if it's
supposed to be an "advanced feature", it should be OK to expect the user
to know (or find out) what paging means.

> I don't really like the extra configuration option for pagingauto but I
> think pagingauto and mem-{paging,footprint}-set should be considered
> advanced options and by default we would recommend that folks just set
> "paging=on" and use mem-set. It should be reasonably clear to users that
> if they disable auto mode then they are expected to understand what is
> happening sufficiently to make their own choices about paging targets
> etc.
> 
> We can probably think of more useful algorithms than raw pagingdelay
> (i.e. based on rate of progress or something) which might be useful for
> larger domains making large changes to the balloon -- lets leave that
> aside for now though. Likewise "auto" mode allows scope for us to
> implement improved algorithms in the future.




_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel