Re: [Xen-users] Xen4.0.1 : slow Disk IO on DomU

[Erwan RENIER <erwan.renier@xxxxxxxxxxx>]

Le 17/03/2011 09:31, Joost Roeleveld a écrit :
> On Wednesday 16 March 2011 23:31:31 Erwan RENIER wrote:
> > Hi,
> > When i test the IO bandwidth it's pretty much slower on DomU :
> >
> > Dom0 read  : 180MB/s write  : 60MB/s
> > DomU read  : 40MB/s write :  6MB/s
> Just did the same tests on my installation (not yet on Xen4):
> Dom0:
> # hdparm -Tt /dev/md5
> /dev/md5:
>   Timing cached reads:   6790 MB in  1.99 seconds = 3403.52 MB/sec
>   Timing buffered disk reads:  1294 MB in  3.00 seconds = 430.94 MB/sec
>
> (md5 = 6-disk RAID-5 software raid)
>
> # hdparm -Tt /dev/vg/domU_sdb1
> /dev/vgvg/domU_sdb1:
>   Timing cached reads:   6170 MB in  2.00 seconds = 3091.21 MB/sec
>   Timing buffered disk reads:  1222 MB in  3.00 seconds = 407.24 MB/sec
>
> DomU:
> # hdparm -Tt /dev/sdb1
> /dev/sdb1:
>   Timing cached reads:   7504 MB in  1.99 seconds = 3761.93 MB/sec
>   Timing buffered disk reads:  792 MB in  3.00 seconds = 263.98 MB/sec
>
> Like you, I do see some drop in performance, but not as severe as you are
> experiencing.
>
> > DomU disks are Dom0 logical volumes, i use paravirtualized guests, the
> > fs type is ext4.
> How do you pass the disks to the domU?
> I pass them as such:
> disk = ['phy:vg/domU_sda1,sda1,w',
> (rest of the partitions removed for clarity)
My DomU conf is like this :
kernel =  "vmlinuz-2.6.32-5-xen-amd64"
ramdisk = "initrd.img-2.6.32-5-xen-amd64"
root = "/dev/mapper/pvops-root"
memory = "512"
disk = [ 'phy:vg0/p2p,xvda,w' , 'phy:vg0/mmd,xvdb1,w', 'phy:sde3,xvdb1,w' ]
vif = [ 'bridge=eth0' ]
vfb = [ 'type=vnc,vnclisten=0.0.0.0' ]
keymap = 'fr'
serial = 'pty'
vcpus = 2
on_reboot = 'restart'
on_crash = 'restart'

> > I already tried the ext4 options barrier=0,data=writeback , it doesn't
> > realy change anything.
> > I tried too with ext2 and ext3 , it's the same.
> To avoid any "issues" with the filesystem, what does "hdparm -Tt<device>" give
> you?
Dom0 :
/dev/sde( single disk ):
 Timing cached reads:   6086 MB in  2.00 seconds = 3050.54 MB/sec
 Timing buffered disk reads: 270 MB in  3.01 seconds =  89.81 MB/sec
/dev/md127 ( raid 5 of 5 disks ):
 Timing cached reads:   6708 MB in  1.99 seconds = 3362.95 MB/sec
 Timing buffered disk reads: 1092 MB in  3.00 seconds = 363.96 MB/sec

DomU :
/dev/xvda:
 Timing cached reads:   5648 MB in  2.00 seconds = 2830.78 MB/sec
 Timing buffered disk reads:  292 MB in  3.01 seconds =  97.16 MB/sec
/dev/xvda2:
 Timing cached reads:   5542 MB in  2.00 seconds = 2777.66 MB/sec
 Timing buffered disk reads:  274 MB in  3.01 seconds =  90.94 MB/sec
/dev/xvdb1:
 Timing cached reads:   5526 MB in  2.00 seconds = 2769.20 MB/sec
 Timing buffered disk reads:  196 MB in  3.02 seconds =  64.85 MB/sec
/dev/xvdb2:
 Timing cached reads:   5334 MB in  2.00 seconds = 2672.47 MB/sec
 Timing buffered disk reads:  166 MB in  3.03 seconds =  54.70 MB/sec

> > Is this normal ?
> Some drop, yes. loosing 90% performance isn't
>
> > If not what do you think the problem is?
> Either you are hitting a bug or it's a configuration issue.
> What is the configuration for your domU? And specifically the way you pass the
> LVs to the domU.
As you can see :
xvda is a lv exported as a whole disk  with lvm  on it, so xvda2 is a lv 
from a vg  in a lv ( ext4 => lv => vg => pv => virtual disk => lv =>vg 
=>pv => raid5 =>disk )
xvdb1 is a lv exported as a partition ( ext4 => virtual part => lv => vg 
=> pv => raid5 => disk )
xvdb2 is a physical partition  exported as a partition ( ext3 => virtual 
part => disk )
Curiously it seems the more complicated, the better it is :/

Thanks.

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users