[Xen-devel] iSCSI connection corrupts Xen block devices.

["Dr. Greg Wettstein" <greg@xxxxxxxxxxxxxxxxx>]

Hi, hope the week has started out well for everyone.

This report may be in the FWIW department since there may be a
fundamental reason why this doesn't work.  We elected to report this
to the Xen community since we thought any behavior which corrupted
disk images needed to at least be reported.

We are maintaining the Xen-SAN release which provides hotplug
functionality to allow Xen guests to participate as first class
entities in either an iSCSI or fibre-channel storage network.  We were
preparing for a second release and ran across behavior which appears
to cause Xen guest block devices to be corrupted.

Relevant VM/OS versions are as follows:

        dom0:   3.4.35
        domU:   3.4.35
        Xen:    4.2.1

The test environment is a domU VM running which is using SCST
(2.2.0) to export a block device via iSCSI.  An iSCSI connection is
initiated from dom0 to the target VM.  The iSCSI block device has a VM
system image on it.  I/O can be done from dom0 to the guest without
any apparent issues; ie, mounting the filesystem and reading and
writing to it.

The problem occurs when a second VM is started which uses the iSCSI
based block device as its root filesystem.  The VM starts and
functions normally, I/O can be done without any issues from inside the
VM.  When the VM is shutdown and the iSCSI connection is closed the
block device is instantly corrupted.

The corruption isn't subtle with the begining of the block device
being over-written with what appears to be generic contents of the
filesystem.  The corruption doesn't occur when the VM shuts down, only
when the iSCSI connection is closed.

If the iSCSI VM target server is run on a separate physical dom0 host
everything functions normally.  So the corruption is definitely linked
to the both VM's being run on the same physical dom0 instance.

The problem occurs regardless of the type of device backend which is
used for the domU block device exported by SCST.  The behavior has
been verified with blktap, image over loop and qdisk.  The problem
also occurs when either FILEIO or BLOCKIO are used for the SCST
virtual disk.

As I said at the outset exposing a device to blkback twice may be
something it was never designed to do.  That being said using VM's for
this type of testing certainly makes sense and the behavior is
unexpected.

Let us know if there are any questions or if additional testing is
needed.

Have a good remainder of the week.

Greg

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-1686
FAX: 701-281-3949           EMAIL: greg@xxxxxxxxxxxx
------------------------------------------------------------------------------
"Laugh now but you won't be laughing when we find you laying on the
 side of the road dead."
                                -- Betty Wettstein
                                   At the Lake

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel