[Xen-devel] Migration issues with 4.1.

["Dr. Greg Wettstein" <greg@xxxxxxxxxxxxxxxxx>]

Good morning, hope everyone's day is going well.

We have sorted out most of the issues with a new iSCSI hotplug script
which allows Xen guests to be treated as first class SAN guests.  The
script allows each virtual machine to be treated as an independent
initiator (IQN) which enables guests to be managed through LUN masking
initiator groups on popular target platforms such as SCST.

When we began testing live migration on Xen 4.2.1 we noted problems
with PVOPS kernels not starting on the migration target.  The
following is output when a migration is attempted:

---------------------------------------------------------------------------
migration target: Ready to receive domain.
Saving to migration stream new xl format (info 0x0/0x0/8146)
Loading new save file <incoming migration stream> (new xl fmt info 
0x0/0x0/8146) Savefile contains xl domain config
xc: Saving memory: iter 0 (last sent 0 skipped 0): 65536/65536  100%
xc: Saving memory: iter 1 (last sent 65457 skipped 79): 65536/65536  100%
xc: Saving memory: iter 2 (last sent 86 skipped 0): 65536/65536  100%
xc: Saving memory: iter 3 (last sent 16 skipped 0): 65536/65536  100%
migration receiver stream contained unexpected data instead of ready message
(command run was: exec ./xen-migrate rainbow xl migrate-receive )
migration target: Transfer complete, requesting permission to start domain.
libxl: error: libxl_utils.c:363:libxl_read_exactly: file/stream truncated 
reading GO message from migration stream
migration target: Failure, destroying our copy.
migration child [8355] not exiting, no longer waiting (exit status will be 
unreported)
Migration failed, resuming at sender.
---------------------------------------------------------------------------

The xen-migrate script we are using is as follows:

---------------------------------------------------------------------------
#! /bin/bash
exec ssh $1 /usr/sbin/xl migrate-receive;
---------------------------------------------------------------------------

We have verified the problem persists up to and including Linux 3.7.5
but it seems like a toolstack problem rather then anything to do with
the kernel.

The output above indicates the guest is resumed on the sender but the
guest is dead with a kernel panic after the migration attempt.  I
don't have the kernel panic handy but can post it if that would be
helpful, but this issue is really secondary to the migration failure.

An 'xl save' followed by transfer of the image to the target and
re-starting the image with 'xl restore' works perfectly.

I found a reference to a similar problem, including the guest panic,
with 4.0.x but never stumbled on the resolution of the problem.

We will be making the iSCSI script available but since live migration
is the obvious application we want to sort this issue first to verify
we are offering something useful to the community.

Will look forward to any comment/suggestions.

Have a good weekend.

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@xxxxxxxxxxxx
------------------------------------------------------------------------------
"There are two things that are infinite; Human stupidity and the
 universe.  And I'm not sure about the universe."
                                -- Albert Einstein

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel