Re: [Xen-devel] Suggestion for merging xl save/restore/migrate/migrate-receive

[George Dunlap <George.Dunlap@xxxxxxxxxxxxx>]

create ^
title it xl migrate transport improvements
thanks

On Mon, Sep 16, 2013 at 4:51 PM, Zhigang Wang <zhigang.x.wang@xxxxxxxxxx> wrote:
> On 09/16/2013 06:04 AM, George Dunlap wrote:
>> On Fri, Sep 13, 2013 at 5:04 PM, Zhigang Wang <zhigang.x.wang@xxxxxxxxxx> 
>> wrote:
>>> Hi,
>>>
>>> As we talked in
>>> http://lists.xen.org/archives/html/xen-devel/2013-09/msg00211.html , I have 
>>> a
>>> suggestion: what about merging xl save/restore/migrate/migrate-receive?
>>>
>>> Here is the description: xl-migrate.rst
>>
>> Thanks for bringing this up, but this thing with attaching files that
>> actually contain your proposal isn't really going to work.  Can you
>> please resend this with your proposals in-line, so that 1) the entire
>> discussion can happen in the mail reader, rather than having to switch
>> back and forth between an editor and a reader, 2) people can comment
>> in-line on the proposals?
>>
>> Thanks,
>>  -George
>>
>
> Thanks George for the comment. Here it is:
>
> As we talked in
> http://lists.xen.org/archives/html/xen-devel/2013-09/msg00211.html , I have a
> suggestion: what about merging xl save/restore/migrate/migrate-receive?
>
> Here is the description: xl-migrate.rst
>
> Also there is a workaround for the xl migration daemon example:
> xl-migrate-socat.rst
>
> Example patch for the workaround: xen-xl-migrate-socat.patch
>
> (I don't have enough knowledge to implement the merge.)
>
> Thanks,
>
> Zhigang
>
> ---- xl-migrate.rst ----
>
> ==========
> XL Migrate
> ==========
>
> :Date: 2013-09-16
>
> Current Status
> ==============
>
> * xl migrate leverages ssh/sshd.
>
> * In order to migrate a VM without user interactive, we have to configure ssh
>   keys for all Servers in a pool. Key management brings complexity.
>
> * In certain cases, customers need non-ssl migrate, which greatly improves the
>   migration speed. There's no way to do it with ssh.
>
> * Current xl migrate command is not intuitive, especially the `-s` option::
>
>       # xl migrate
>       Usage: xl [-v] migrate [options] <Domain> <host>
>
>       Save a domain state to restore later.
>
>       Options:
>
>       -h              Print this help.
>       -C <config>     Send <config> instead of config file from creation.
>       -s <sshcommand> Use <sshcommand> instead of ssh.  String will be passed
>                       to sh. If empty, run <host> instead of ssh <host> xl
>                       migrate-receive [-d -e]
>       -e              Do not wait in the background (on <host>) for the death
>                       of the domain.
>
>   It's a little hard to adapt other tools as transport.
>
> * We have differnt implementation for `xl save/restore` and
>   `xl migrate/migrate-receive`. Can we merge them?
>
>
> Proposal
> ========
>
> * Implement dedicated daemons for ssl and non-ssl migration receive
>   (`socat <http://www.dest-unreach.org/socat/>`_ can be used).
>
>   Example patch for dedicated migrate receive daemon:
>   xen-xl-migrate-socat.patch
>
>   `socat` will call `xl restore` instead of `xl migrate-receive` after the
>   following change.
>
> * Merge `xl migrate/migrate-receive` to `xl save/restore`:
>
>   - To save a VM::
>
>       # xl save [-c] <domain> -f vm.chk
>
>     Or::
>
>       # xl save [-c] <domain> >vm.chk
>
>   - To restore a VM::
>
>       # xl restore -f vm.chk
>
>     Or::
>
>       # cat vm.chk | xl restore
>
>   - To migrate a VM using ssh/sshd::
>
>       # xl save -c <domain> | ssh root@<remote-host> xl restore
>
>     We can implement a wrapper to make `xl migrate <domain> <remote-host>` to
>     call the above command.
>
>   - To migrate a VM using dedicated migrate receive daemon::
>
>       # xl save -c <domain> | socat - TCP:<remote-host>:8004"
>
>     Or with SSL::
>
>       # xl save -c <domain> | socat - OPENSSL:<remote-host>:8005,verify=0
>
>   - Localhost migration::
>
>       # xl save <domain> | xl restrore
>
>   - Localhost live migration::
>
>       # xl save -c <domain> | xl restrore
>
>   Patch: I don't have the capability and time to implement it yet.
>
>
> ---- xl-migrate-socat.rst ----
>
> ==========
> XL Migrate
> ==========
>
> :Date: 2013-09-16
>
> Current Status
> ==============
>
> * xl migrate leverages ssh/sshd::
>
>       xl migrate <domain> <host>
>
> * In order to migrate a VM without user interactive, we have to configure ssh
>   keys for all Servers in a pool. Key management with dynamic Server Pools is
>   error prone.
>
> * In certain cases, customers need non-ssl migrate, which greatly improves the
>   migration speed. There's no way to do it with ssh.
>
>
> Proposal
> ========
>
> * Implement dedicated daemons for ssl and non-ssl migration receive.
>
> * `socat <http://www.dest-unreach.org/socat/>`_ can be used.
>
> * Patch: xen-xl-migrate-socat.patch
>
>
> Testing
> =======
>
> * Install the new Xen.
>
> * Start the services::
>
>       service xl-migrate-received start
>       service xl-ssl-migrate-received start
>
> * Start a VM::
>
>       xl create vm.cfg
>
> * Migrate::
>
>       xl migrate -s xl-migrate-command <domid> localhost
>
> * SSL migrate::
>
>       xl migrate -s xl-ssl-migrate-command <domid> localhost
>
> * Speed for PV guest with 1024M memory:
>
>   - ssh: 45s
>   - socat: 9s
>   - socat ssl: 57s
>
>
> ---- xen-xl-migrate-socat.patch ----
>
> diff --git a/tools/hotplug/Linux/Makefile b/tools/hotplug/Linux/Makefile
> index 47655f6..586728b 100644
> --- a/tools/hotplug/Linux/Makefile
> +++ b/tools/hotplug/Linux/Makefile
> @@ -55,6 +55,8 @@ endif
>         $(INSTALL_PROG) $(XENCOMMONS_INITD) $(DESTDIR)$(INITD_DIR)
>         $(INSTALL_DATA) $(XENCOMMONS_SYSCONFIG) 
> $(DESTDIR)$(SYSCONFIG_DIR)/xencommons
>         $(INSTALL_PROG) init.d/xen-watchdog $(DESTDIR)$(INITD_DIR)
> +       $(INSTALL_PROG) init.d/xl-migrate-received $(DESTDIR)$(INITD_DIR)
> +       $(INSTALL_PROG) init.d/xl-ssl-migrate-received $(DESTDIR)$(INITD_DIR)
>
>  .PHONY: install-scripts
>  install-scripts:
> diff --git a/tools/hotplug/Linux/init.d/xl-migrate-received 
> b/tools/hotplug/Linux/init.d/xl-migrate-received
> new file mode 100644
> index 0000000..d67cf7d
> --- /dev/null
> +++ b/tools/hotplug/Linux/init.d/xl-migrate-received
> @@ -0,0 +1,106 @@
> +#!/bin/bash
> +#
> +# xl-migrate-received: xl migrate receive daemon
> +#
> +# chkconfig: 2345 80 20
> +# description: xl migrate receive daemon
> +
> +. /etc/rc.d/init.d/functions
> +
> +start() {
> +    socat -ly TCP-LISTEN:8004,reuseaddr,fork EXEC:"xl migrate-receive" 
> >/dev/null 2>&1 </dev/null &
> +    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep 
> | awk '{print $1}')
> +    if [ -n "$pid" ]; then
> +        action $"Starting xl migrate receive daemon: " true
> +        touch /var/lock/subsys/xl-migrate-received
> +    else
> +        action $"Starting xl migrate receive daemon: " false
> +    fi
> +}
> +
> +stop() {
> +    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep 
> | awk '{print $1}')
> +    action $"Stopping xl migrate receive daemon: " kill -9 $pid
> +    [ $? -eq 0 ] && rm -f /var/lock/subsys/xl-migrate-received
> +}
> +
> +status() {
> +    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep 
> | awk '{print $1}')
> +    if [ -n "$pid" ]; then
> +        echo $"xl migrate receive daemon is running"
> +    else
> +        echo $"xl migrate receive daemon is stopped"
> +    fi
> +}
> +
> +case "$1" in
> +    start)
> +        start
> +        ;;
> +    stop)
> +        stop
> +        ;;
> +    restart)
> +        stop
> +        start
> +        ;;
> +    status)
> +        status
> +        ;;
> +    *)
> +        echo $"Usage: $0 {start|stop|restart|status}"
> +        exit 1
> +esac
> +#!/bin/bash
> +#
> +# xl-migrate-received: xl migrate receive daemon
> +#
> +# chkconfig: 2345 80 20
> +# description: xl migrate receive daemon
> +
> +. /etc/rc.d/init.d/functions
> +
> +start() {
> +    socat -ly TCP-LISTEN:8004,reuseaddr,fork EXEC:"xl migrate-receive" 
> >/dev/null 2>&1 </dev/null &
> +    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep 
> | awk '{print $1}')
> +    if [ -n "$pid" ]; then
> +        action $"Starting xl migrate receive daemon: " true
> +        touch /var/lock/subsys/xl-migrate-received
> +    else
> +        action $"Starting xl migrate receive daemon: " false
> +    fi
> +}
> +
> +stop() {
> +    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep 
> | awk '{print $1}')
> +    action $"Stopping xl migrate receive daemon: " kill -9 $pid
> +    [ $? -eq 0 ] && rm -f /var/lock/subsys/xl-migrate-received
> +}
> +
> +status() {
> +    pid=$(ps -eo pid,args | grep "socat -ly TCP-LISTEN:8004" | grep -v grep 
> | awk '{print $1}')
> +    if [ -n "$pid" ]; then
> +        echo $"xl migrate receive daemon is running"
> +    else
> +        echo $"xl migrate receive daemon is stopped"
> +    fi
> +}
> +
> +case "$1" in
> +    start)
> +        start
> +        ;;
> +    stop)
> +        stop
> +        ;;
> +    restart)
> +        stop
> +        start
> +        ;;
> +    status)
> +        status
> +        ;;
> +    *)
> +        echo $"Usage: $0 {start|stop|restart|status}"
> +        exit 1
> +esac
> diff --git a/tools/hotplug/Linux/init.d/xl-ssl-migrate-received 
> b/tools/hotplug/Linux/init.d/xl-ssl-migrate-received
> new file mode 100644
> index 0000000..272ed9c
> --- /dev/null
> +++ b/tools/hotplug/Linux/init.d/xl-ssl-migrate-received
> @@ -0,0 +1,106 @@
> +#!/bin/bash
> +#
> +# xl-ssl-migrate-received: xl ssl migrate receive daemon
> +#
> +# chkconfig: 2345 80 20
> +# description: xl ssl migrate receive daemon
> +
> +. /etc/rc.d/init.d/functions
> +
> +start() {
> +    socat -ly 
> OPENSSL-LISTEN:8005,reuseaddr,fork,key=/etc/ovs-agent/cert/key.pem,cert=/etc/ovs-agent/cert/certificate.pem,verify=0
>  EXEC:"xl migrate-receive" >/dev/null 2>&1 </dev/null &
> +    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v 
> grep | awk '{print $1}')
> +    if [ -n "$pid" ]; then
> +        action $"Starting xl ssl migrate receive daemon: " true
> +        touch /var/lock/subsys/xl-migrate-received
> +    else
> +        action $"Starting xl ssl migrate receive daemon: " false
> +    fi
> +}
> +
> +stop() {
> +    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v 
> grep | awk '{print $1}')
> +    action $"Stopping xl ssl migrate receive daemon: " kill -9 $pid
> +    [ $? -eq 0 ] && rm -f /var/lock/subsys/xl-migrate-received
> +}
> +
> +status() {
> +    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v 
> grep | awk '{print $1}')
> +    if [ -n "$pid" ]; then
> +        echo $"xl ssl migrate receive daemon is running"
> +    else
> +        echo $"xl ssl migrate receive daemon is stopped"
> +    fi
> +}
> +
> +case "$1" in
> +    start)
> +        start
> +        ;;
> +    stop)
> +        stop
> +        ;;
> +    restart)
> +        stop
> +        start
> +        ;;
> +    status)
> +        status
> +        ;;
> +    *)
> +        echo $"Usage: $0 {start|stop|restart|status}"
> +        exit 1
> +esac
> +#!/bin/bash
> +#
> +# xl-ssl-migrate-received: xl ssl migrate receive daemon
> +#
> +# chkconfig: 2345 80 20
> +# description: xl ssl migrate receive daemon
> +
> +. /etc/rc.d/init.d/functions
> +
> +start() {
> +    socat -ly 
> OPENSSL-LISTEN:8005,reuseaddr,fork,key=/etc/ovs-agent/cert/key.pem,cert=/etc/ovs-agent/cert/certificate.pem,verify=0
>  EXEC:"xl migrate-receive" >/dev/null 2>&1 </dev/null &
> +    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v 
> grep | awk '{print $1}')
> +    if [ -n "$pid" ]; then
> +        action $"Starting xl ssl migrate receive daemon: " true
> +        touch /var/lock/subsys/xl-migrate-received
> +    else
> +        action $"Starting xl ssl migrate receive daemon: " false
> +    fi
> +}
> +
> +stop() {
> +    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v 
> grep | awk '{print $1}')
> +    action $"Stopping xl ssl migrate receive daemon: " kill -9 $pid
> +    [ $? -eq 0 ] && rm -f /var/lock/subsys/xl-migrate-received
> +}
> +
> +status() {
> +    pid=$(ps -eo pid,args | grep "socat -ly OPENSSL-LISTEN:8005" | grep -v 
> grep | awk '{print $1}')
> +    if [ -n "$pid" ]; then
> +        echo $"xl ssl migrate receive daemon is running"
> +    else
> +        echo $"xl ssl migrate receive daemon is stopped"
> +    fi
> +}
> +
> +case "$1" in
> +    start)
> +        start
> +        ;;
> +    stop)
> +        stop
> +        ;;
> +    restart)
> +        stop
> +        start
> +        ;;
> +    status)
> +        status
> +        ;;
> +    *)
> +        echo $"Usage: $0 {start|stop|restart|status}"
> +        exit 1
> +esac
> diff --git a/tools/misc/Makefile b/tools/misc/Makefile
> index 59def7a..88aaf20 100644
> --- a/tools/misc/Makefile
> +++ b/tools/misc/Makefile
> @@ -16,7 +16,7 @@ TARGETS := $(TARGETS-y)
>
>  SUBDIRS := $(SUBDIRS-y)
>
> -INSTALL_BIN-y := xencons xencov_split
> +INSTALL_BIN-y := xencons xencov_split xl-migrate-command 
> xl-ssl-migrate-command
>  INSTALL_BIN-$(CONFIG_X86) += xen-detect
>  INSTALL_BIN := $(INSTALL_BIN-y)
>
> diff --git a/tools/misc/xl-migrate-command b/tools/misc/xl-migrate-command
> new file mode 100644
> index 0000000..0558ff9
> --- /dev/null
> +++ b/tools/misc/xl-migrate-command
> @@ -0,0 +1,3 @@
> +#!/bin/sh
> +
> +socat - TCP:$1:8004
> diff --git a/tools/misc/xl-ssl-migrate-command 
> b/tools/misc/xl-ssl-migrate-command
> new file mode 100644
> index 0000000..628c554
> --- /dev/null
> +++ b/tools/misc/xl-ssl-migrate-command
> @@ -0,0 +1,3 @@
> +#!/bin/sh
> +
> +socat - OPENSSL:$1:8005,verify=0
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel