[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Benchmark Xen writes with sync - Xen ignores fsync, O_SYNC?

On 6 April 2012 08:30, Pablo Llopis <pllopis@xxxxxxxxxxxxxxxxx> wrote:
> Hi there Xen community,
> I am trying to benchmark and compare I/O in Xen/domU to native performance.
> In order to do this I started trying to benchmark writes so as to avoid
> caching effects that surely turn up when performing reads due to the page
> cache et al.
> However, I have quickly run into a problem: Xen domU reports that a 128MB
> file is written at close to 300MB/s, while the disk's performance peaks at
> about 80MB/s (I observed this on a dom0 and on a bare-metal kernel with no
> hypervisor).
> Please note that I fsync() after all writes in hopes to avoid the effect of
> write buffers. I have tried with O_SYNC as well, observing a similar
> outcome.
> I can confirm this writing a simple program, and verified exactly same
> results running bonnie++ with the fsync() option turned on.
> I am surprised to see writes reaching a throughput as high as 300MB/s, as
> the disk surely isn't physically capable of reaching that bandwidth, meaning
> that writes are not being really synced to disk.
> Is this a bug in Xen, or is there a way to make Xen not ignore fsync,
> fdatasync, O_SYNC, etc..?
> How would I proceed to measure and compare real read/write speeds on a Xen
> domU ?
> My disk drivers are specified with "file:/path/to/image.img,xvda,1,w" (I
> could not get the tapdisk driver to work properly, I tried with vanilla 3.2
> and 3.0.0 ubuntu kernels)

To measure real disk speed I recommend using the phy:/ handler with a
block device.
You can use a partition, an LVM logical volume or an entire disk.
Use of files on filesytems for domain block devices is discouraged and
should be avoided at all costs.

For testing IO throughput actively I wouldn't use bonnie++ as it is a
filesystem level tool and not overly accurate at that.
Use the Flexible IO Tester (aka fio) by Jens Axobe.
You can install it on Ubuntu with:
apt-get install fio

I would also recommend using the libaio engine with fio if you are
using an SSD and need to drive the queue depth up to get a better idea
of the IOPs the device can deliver.
You will need to install libaio with:
apt-get install libaio

> Xen is version 4.1.1 and is running Oneiric domUs (kernel 3.0.0)
> For the dom0 I have a 3.2 vanilla kernel and a ubuntu (oneiric) 3.0.0 kernel
> Thank you in advance,
> Pablo
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel


Founder | Director | VP Research
Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56
99 52 | Mobile: 0428 754 846

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.