[Xen-users] GPLPV Disk performance block device vs. file based backend

[Dion Kant <dion@xxxxxxxxxx>]

Dear all,

I have two Windows 2008 VMs with latest gplpv drivers installed. One is
based on a file container

name="wsrv-file"
disk=[ 'file:/var/lib/xen/images/wsrv-file/disk0.raw,xvda,w', ]

and the other one is based on a block device.

name="wsrv-bd"
disk=[ 'phy:/dev/vg0/wsrv-bd,xvda,w', ]

The block device represents a logical volume consisting of 4 disks and
is configured with 4 stripes. The file lives on another logical volume
in the same volume group and is formatted with XFS.

Now I measure more than a factor 3 better I/O performance on the file
based VM as compared to the block device based VM. I don't think it is a
cache issue which is tricking me. Details of my test and setup are given
below.

Hardware:
Supermicro X8DTU
2x CPU E5620
RAM 12 GB (6x2GB DIMM)
Disks: 4x 2TB
Controller (Adaptec) SMC AOC-USAS-S4i (driver: aacraid)
Disk setup JBOD

Software:
openSuSE 12.2 x86_64 distribution00000
kernel: 3.4.6-2.10-xen
Xen: 4.1.3_01-5.6.200000

dom0 lives on a 80 GB RAID10 at the outside of the disks, the remaining
partitions are directly (no raid) to a volume group.
The VM disks live on logical volumes in that volume group.

Both Windows VMs are similar Windows Server 2008 R2 Enterprise all
updates installed. They are assigned 4 cpus (no pinning) and 4 GB RAM.

Test setup:
When running a test, only one VM is running, no other things are
happening (just the test) .
Installed Cygwin and use dd for writing to disk:

sync; time (dd if=/dev/zero of=test.bin bs=4096 count=5000000; sync)

Measured time is used to calculate streaming rate, but this could hide
caching in dom0.  However, while running the test, in dom0 a vmstat -n 1
is running so the actual write performance to the block devices can be
observed in dom0. Note we are not looking for some marginal effect!

I noticed that for the file based VM, the "bo" results from vmstat are
doubled, i.e. the bytes written to the disk file living on the LV are
counted twice. This does not happen for a block device based disk.
00000
block device based: 150 MB/s
file based: >450 MB/s

I have reproduced this behaviour using previous versions of openSUSE
(older Xen) and other Windows versions (XP, 2003). Furthermore I
observed this behaviour on other hardware, mostly tested this on
Supermicro mother boards.

I can provide more details if required and I can do more testing as well.

Regards,

Dion


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users