Re: [Xen-devel] Benchmark Xen writes with sync - Xen ignores fsync, O_SYNC?

[Joseph Glanville <joseph.glanville@xxxxxxxxxxxxxx>]

On 6 April 2012 08:30, Pablo Llopis <pllopis@xxxxxxxxxxxxxxxxx> wrote:
> Hi there Xen community,
>
> I am trying to benchmark and compare I/O in Xen/domU to native performance.
>
> In order to do this I started trying to benchmark writes so as to avoid
> caching effects that surely turn up when performing reads due to the page
> cache et al.
> However, I have quickly run into a problem: Xen domU reports that a 128MB
> file is written at close to 300MB/s, while the disk's performance peaks at
> about 80MB/s (I observed this on a dom0 and on a bare-metal kernel with no
> hypervisor).
> Please note that I fsync() after all writes in hopes to avoid the effect of
> write buffers. I have tried with O_SYNC as well, observing a similar
> outcome.
> I can confirm this writing a simple program, and verified exactly same
> results running bonnie++ with the fsync() option turned on.
>
> I am surprised to see writes reaching a throughput as high as 300MB/s, as
> the disk surely isn't physically capable of reaching that bandwidth, meaning
> that writes are not being really synced to disk.
>
> Is this a bug in Xen, or is there a way to make Xen not ignore fsync,
> fdatasync, O_SYNC, etc..?
> How would I proceed to measure and compare real read/write speeds on a Xen
> domU ?
>
> My disk drivers are specified with "file:/path/to/image.img,xvda,1,w" (I
> could not get the tapdisk driver to work properly, I tried with vanilla 3.2
> and 3.0.0 ubuntu kernels)

To measure real disk speed I recommend using the phy:/ handler with a
block device.
You can use a partition, an LVM logical volume or an entire disk.
Use of files on filesytems for domain block devices is discouraged and
should be avoided at all costs.

For testing IO throughput actively I wouldn't use bonnie++ as it is a
filesystem level tool and not overly accurate at that.
Use the Flexible IO Tester (aka fio) by Jens Axobe.
You can install it on Ubuntu with:
apt-get install fio

I would also recommend using the libaio engine with fio if you are
using an SSD and need to drive the queue depth up to get a better idea
of the IOPs the device can deliver.
You will need to install libaio with:
apt-get install libaio

> Xen is version 4.1.1 and is running Oneiric domUs (kernel 3.0.0)
> For the dom0 I have a 3.2 vanilla kernel and a ubuntu (oneiric) 3.0.0 kernel
>
> Thank you in advance,
>
> Pablo
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel
>

Joseph.

-- 
Founder | Director | VP Research
Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56
99 52 | Mobile: 0428 754 846

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel