Re: [Xen-users] Disk IO trouble in Xen

[Greg Hellings <greg.hellings@xxxxxxxxxxxx>]

So I tracked down my network freezing issues.  I have a few xen servers all
connected to the same network I'm working with here. All of them I used the
virt-manager tool to install.  One of the other hypervisors had a domu
running on it that virt-manager had chosen the same mac address as the domu
on the hypervisor I was having problems with.

I still see the same performance difference between the dom0 and domu when
using hdparm -tT, but maybe that isn't a good test.  I do notice that if I
run hdparm over and over on the domu's the numbers improve, suggesting that
caching is effecting the out come.  Under iozone, the number between domu
and the dom0 are almost exactly the same.

--
Greg


On 5/5/08 2:58 PM, "Ross S. W. Walker" <rwalker@xxxxxxxxxxxxx> wrote:

> Greg Hellings wrote:
>> 
>> I started this thinking I was having an IO problem.  Now, I'm
>> not so sure.
>> I am using a pre-allocated disk, not a sparse file.
>> 
>> Write performance seems about the same for both
>> 
>> Dom0
>> [root@localhost ~]# dd if=/dev/zero of=/tmp/dummy count=2097152
>> 2097152+0 records in
>> 2097152+0 records out
>> 1073741824 bytes (1.1 GB) copied, 29.8368 seconds, 36.0 MB/s
>> 
>> DomU
>> [root@localhost ~]# dd if=/dev/zero of=/tmp/dummy count=2097152
>> 2097152+0 records in
>> 2097152+0 records out
>> 1073741824 bytes (1.1 GB) copied, 30.4716 seconds, 35.2 MB/s
>> 
>> Iozone shows very similar numbers on both the Dom0 and DomU. The DomU seems
>> to freeze periodically and drop network connections.  I assumed this was IO
>> related because everything else seems fine and those hdparm numbers are so
>> far off.  Also heavy IO related tasks seem slow.  I'm fairly baffled now.
>> 
> 
> Run a 'vmstat 1' and take a look at the interrupts and io waits.
> 
> Maybe the network is generating a high load?
> 
> Performance, and performance under load are two different things.
> 
> 
>> On 5/2/08 4:04 AM, "Sadique Puthen" <sputhenp@xxxxxxxxxx> wrote:
>> 
>>> Are you using a pre-allocated image or a sparse file? If
>> the latter it's
>>> likely to have performance problems like you found?  What
>> if you test
>>> with dd or iozone or bonnie++?
>>> 
>>> --Sadique
>>> 
>>> Greg Hellings wrote:
>>>> I'm having terrible IO trouble with an image based domu
>>>> 
>>>> 
>>>> This is the performance on the Dom0
>>>> [root@localhost /]# hdparm -tT /dev/sda
>>>> 
>>>> /dev/sda:
>>>>  Timing cached reads:   3228 MB in  2.00 seconds = 1617.22 MB/sec
>>>>  Timing buffered disk reads:  174 MB in  3.02 seconds =
>> 57.60 MB/sec
>>>> 
>>>> 
>>>> And this is the performance on the DomU
>>>> [root@localhost ~]# hdparm -tT /dev/xvda
>>>> 
>>>> /dev/xvda:
>>>>  Timing cached reads:   3336 MB in  2.00 seconds = 1669.98 MB/sec
>>>>  Timing buffered disk reads:   26 MB in  3.13 seconds =
>> 8.30 MB/sec
>>>> 
>>>> 
>>>> Is it normal for the performance difference to be so
>> great?  I'm running
>>>> CentOS 5 and the DomU image is stored on a LVM ReiserFS
>> formatted partition.
>>>> 
>>>> Here is the config for the DomU
>>>> name = "dddevdb"
>>>> uuid = "927f1e915813c2dec60c8f76c2716783"
>>>> maxmem = 3072
>>>> memory = 2042
>>>> vcpus = 2
>>>> bootloader = "/usr/bin/pygrub"
>>>> on_poweroff = "destroy"
>>>> on_reboot = "restart"
>>>> on_crash = "restart"
>>>> vfb = [ "type=vnc,vncunused=1,keymap=en-us" ]
>>>> disk = [ "tap:aio:/var/lib/xen/images/dddevdb.img,xvda,w"]
>>>> vif = [ "mac=00:16:3e:62:7a:d1,bridge=xenbr0" ]
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Xen-users mailing list
>>>> Xen-users@xxxxxxxxxxxxxxxxxxx
>>>> http://lists.xensource.com/xen-users
>>>> 
>>>>   
>>> 
>> 
>> 
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-users
>> 
> 
> ______________________________________________________________________
> This e-mail, and any attachments thereto, is intended only for use by
> the addressee(s) named herein and may contain legally privileged
> and/or confidential information. If you are not the intended recipient
> of this e-mail, you are hereby notified that any dissemination,
> distribution or copying of this e-mail, and any attachments thereto,
> is strictly prohibited. If you have received this e-mail in error,
> please immediately notify the sender and permanently delete the
> original and any copy or printout thereof.
> 


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users