Re: [Xen-users] linux stubdom

[Roger Pau Monnà <roger.pau@xxxxxxxxxx>]

On 11/02/13 16:05, Ian Campbell wrote:
> On Mon, 2013-02-11 at 14:59 +0000, Markus Hochholdinger wrote:
>> Hello,
>>
>> Am 06.02.2013 um 15:39 Uhr schrieb Markus Hochholdinger 
>> <Markus@xxxxxxxxxxxxxxxxx>:
>>> Am 01.02.2013 um 09:56 Uhr schrieb Ian Campbell <Ian.Campbell@xxxxxxxxxx>:
>> [..]
>>>>> executed? Would this be between #3 and #7?
>>>> It is part of the device teardown and setup, so it is during #5 and #6
>>>> (strictly I think it is just after #5 and just before #6).
>>>> On xen-devel at the minute there is a patch series under discussion to
>>>> make the script hooks more flexible, in particular adding pre and post
>>>> migrate hooks (called something like #1-#3 and #7-#7) which can pre
>>>> setup bits of the storage stack which are safe to do with the guest
>>>> running but might be slow to initialise (e.g. iSCSI login, but not
>>>> opening the device). I don't think this needs to affect you though.
>>> at least with xm toolstack, block-scripts in /etc/xen/scripts/block-* are
>>> executed on the destination host (add) before the script (remove) on the
>>> source host is executed while live migrating a domU.
>>> (I've created the script /etc/xen/scripts/block-md which assembles and
>>> stops raid1 devices.)
>>> Next, I'll test xl toolstack with this setup.
>>
>> with Xen 4.1 there's no support for custom scripts within libxl.
>>
>> With Xen 4.2.1 there's support for custom scripts (like with xm toolstack) 
>> but 
>> also only add/remove. And add is called on the destination side before 
>> remove 
>> is called on the transmitting side while doing a live migration of a domU.

Yes, I've also realized this while working on the new hotplug
implementation. The hotplug script is executed on the destination before
the other end has executed the remove script (this is due to the fact
that the remove script is executed when the migrated domain is destroyed
on the source). So at a certain point the destination host has executed
the "add" script before the source host executes the "remove" hotplug
script.

This is not a problem with the current hotplug scripts in-three, because
we can guarantee that the device will not be accessed simultaneously
(because the guest only resumes on either the source or the destination
hosts, but never on both).

So the scheme looks more like:

     1. First an empty shell domain (with no devices) is created on the
        target host.
     2. Then we copy the memory over, in several iterations, while the
        domain is running on the source host (iterations happen to
        handle the guest dirtying memory as we copy, this is the "live"
        aspect of the migration).
     3. After some iterations of live migration we pause the source
        guest
     4. Setup devices on the target host for the incoming domain
     5. Now we copy the remaining dirty RAM
     6. Resume the guest on the target domain
     7. Tear down devices on the source host
     8. Guest reconnects to new backend
     (#7 and #8 can happen in different order)

#4 will be where the hotplug script "add" call happens on the target
host, and #7 where the hotplug script "remove" call happens on the
source host.

> 
> This sounds like a bug which ought to be addressed (Roger, can you take
> a look?)

I think this is how migration works in both xl and xm, but if there are
hotplug scripts that cannot be executed simultaneously (ie you cannot
make two simultaneous calls to "add" without calling "remove" first) we
could mark it as a bug.

It would make the resume on source host more complicated, since in case
of failure we will have to remove the devices on the destination host
and reconnect them on the source host.

>> Next I'll test latest development version...
> 
> I'm not sure it will differ from 4.2.x in this area (yet). Roger can
> probably advise better than me though.

No, this has not changed in -unstable.


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users