Re: [Xen-devel] Fwd: VM Live Migration with Local Storage

[Igor Druzhinin <igor.druzhinin@xxxxxxxxxx>]

On 12/06/17 04:16, Bruno Alvisio wrote:
> Hello,
> 
> I think it would be beneficial to add local disk migration feature for
> ‘blkback' backend since it is one of the mostly used backends. I would
> like to start a discussion about the design of the machinery needed to
> achieve this feature.
> 
> ===========================
> Objective
> Add a feature to migrate VMs that have local storage and use the blkback
> iface.
> ===========================
> 
> ===========================
> User Interface
> Add a cmd line option in “xl migrate” command to specify if local disks
> need to be copied to the destination node.
> ===========================
> 
> ===========================
> Design
> 
>  1. As part of the libxl_domain_suspend, the “disk mirroring machinery”
>     starts an asynchronous job that copies the disks blocks from source
>     to the destination.
>  2. The protocol to copy the disks should resemble the one used for
>     memory copy:
> 
>   * Do first initial copy of the disk.
>   * Check of sectors that have been written since copy started. For
>     this, the blkback driver should be aware that migration of disk is
>     happening and in this case forward the write request to
>     the “migration machinery” so that a record of dirty blocks are logged.
>   * Migration machinery copies “dirty” blocks until convergence.

Be careful with that. You don't really want to merge block and memory
live migrations. They should be linked but proceed independently since
we don't want to have the last iteration of memory transfer stalled
waiting for disk convergence. Some mix of pre-copy and post-copy
approach might be suitable.

Igor

>   * Duplicate all the disk writes/reads to both disks in source and
>     destinations node while VM is being suspended.
> 
> 
> Block Diagram
> 
>    +—------+
>    |  VM   |
>    +-------+
>       |
>       | I/O Write
>       |
>       V
> +----------+       +-----------+                   +-------------+
> |  blkback | ----> |  Source   |  sectors Stream   | Destination |
> +----------+       |  mirror   |------------------>|   mirror    |
>       |            | machinery |   I/O Writes      |  machinery  |
>       |            +-----------+                   +-------------+
>       |                                                  |
>       |                                                  |
>       | To I/O block layer                               |
>       |                                                  |
>       V                                                  V
> +----------+                                       +-------------+
> |   disk   |                                       |   Mirrored  |
> +----------+                                       |     Disk    |
>                                                    +-------------+
> 
> 
> ======================
> Initial Questions
> 
>  1. Is it possible to leverage the current design of QEMU for drive
>     mirroring for Xen?
>  2. What is the best place to implement this protocol? As part of Xen or
>     the kernel?
>  3. Is it possible to use the same stream currently used for migrating
>     the memory to also migrate the disk blocks?
> 
> 
> Any guidance/feedback for a more specific design is greatly appreciated.
> 
> Thanks,
> 
> Bruno
> 
> On Wed, Feb 22, 2017 at 5:00 AM, Wei Liu <wei.liu2@xxxxxxxxxx
> <mailto:wei.liu2@xxxxxxxxxx>> wrote:
> 
>     Hi Bruno
> 
>     Thanks for your interest.
> 
>     On Tue, Feb 21, 2017 at 10:34:45AM -0800, Bruno Alvisio wrote:
>     > Hello,
>     >
>     > I have been to doing some research and as far as I know XEN supports
>     > Live Migration
>     > of VMs that only have shared storage. (i.e. iSCSI) If the VM has been
>     > booted with local storage it cannot be live migrated.
>     > QEMU seems to support live migration with local storage (I have tested 
> using
>     > 'virsh migrate with the '--storage-copy-all' option)
>     >
>     > I am wondering if this still true in the latest XEN release. Are there 
> plans
>     > to add this functionality in future releases? I would be interested in
>     > contributing to the Xen Project by adding this functionality.
>     >
> 
>     No plan at the moment.
> 
>     Xen supports a wide variety of disk backends. QEMU is one of them. The
>     others are blktap (not upstreamed yet) and in-kernel blkback. The latter
>     two don't have the capability to copy local storage to the remote end.
> 
>     That said, I think it would be valuable to have such capability for QEMU
>     backed disks. We also need to design the machinery so that other
>     backends can be made to do the same thing in the future.
> 
>     If you want to undertake this project, I suggest you setup a Xen system,
>     read xl / libxl source code under tools directory and understand how
>     everything is put together. Reading source code could be daunting at
>     times, so don't hesitate to ask for pointers. After you have the big
>     picture in mind, we can then discuss how to implement the functionality
>     on xen-devel.
> 
>     Does this sound good to you?
> 
>     Wei.
> 
>     > Thanks,
>     >
>     > Bruno
> 
>     > _______________________________________________
>     > Xen-devel mailing list
>     > Xen-devel@xxxxxxxxxxxxx <mailto:Xen-devel@xxxxxxxxxxxxx>
>     > https://lists.xen.org/xen-devel <https://lists.xen.org/xen-devel>
> 
> 
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> https://lists.xen.org/xen-devel
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel