|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-users] Disk-I/O Performance DomU vs Dom0
On 6 April 2012 20:15, Ulrich Hochholdinger <uhochholdinger@xxxxxxxxxxxxx> wrote: > Hi, > I experience massive I/O performance-difference between Dom0 and DomU. To > simply show the impact, I use a logical volume striped over 6 hard-drives. > Then (write/read) enough bytes linear via dd zeros (on/from) this lv and > compare the time. > The result: inside DomU I get only about 30% of the I/O performance of the > Dom0. > > So, where is the handbrake, and how do I release it? Normaly there shouldbe > no such big difference. If you need more information about setup and config, > to give me a hint where else to search for my missing 70% performance, simply > reply inside the list. > > Here are the gory details so far: > > Preparations > ================================================================================================== > Dom0: akaar > DomU: guinan > > Create LV: > lvcreate --name=performance-test --size=20G --stripes=6 vg0 > > Attach LV to domU: > root@akaar:~# xm block-attach guinan phy:/dev/vg0/performance-test xvdc1 w > > Test: > ===================================================================================================== > ============== Write > === Dom0 > root@akaar:~# date ; dd if=/dev/zero of=/dev/vg0/performance-test bs=1M > count=4000 ; sync ; date > Do 5. Apr 20:15:50 CEST 2012 > 4000+0 records in > 4000+0 records out > 4194304000 bytes (4,2 GB) copied, 7,34972 s, 571 MB/s > Do 5. Apr 20:15:58 CEST 2012 > > === DomU > root@guinan:~# date ; dd if=/dev/zero of=/dev/xvdc1 bs=1M count=4000 ; sync ; > date > Do 5. Apr 20:34:50 CEST 2012 > 4000+0 records in > 4000+0 records out > 4194304000 bytes (4,2 GB) copied, 23,5331 s, 178 MB/s > Do 5. Apr 20:35:13 CEST 2012 > > > ============== Read > === Dom0 > root@akaar:~# date ; dd if=/dev/vg0/performance-test of=/dev/null bs=1M > count=4000 ; sync ; date > Do 5. Apr 20:21:38 CEST 2012 > 4000+0 records in > 4000+0 records out > 4194304000 bytes (4,2 GB) copied, 8,98257 s, 467 MB/s > Do 5. Apr 20:21:48 CEST 2012 > > === DomU > root@guinan:~# date ; dd if=/dev/xvdc1 of=/dev/null bs=1M count=4000 ; sync ; > date > Do 5. Apr 20:38:04 CEST 2012 > 4000+0 records in > 4000+0 records out > 4194304000 bytes (4,2 GB) copied, 17,8235 s, 235 MB/s > Do 5. Apr 20:38:22 CEST 2012 > ================================================================================================== > ================================================================================================== > > > ================================================================================================== > Technical Data: > =================================================================================================== > Hardware Poweredge 510 / 96GB RAM / 2x X5650 > > Storage 6 Nearline SAS Harddrives attached to Dell Perc H200: > > lvcreate --name=performance-test --size=20G --stripes=6 vg0 > > Dom0 > =================================================================================================== > Default Debian Squeeze 32 Bit Kernel 64Bit HV: > 1GB Ram / fixed / ballooning disabled / all cpus / no pinning > > root@akaar:~# uname -a > Linux akaar 2.6.32-5-xen-686 #1 SMP Mon Mar 26 10:32:49 UTC 2012 i686 > GNU/Linux > > > root@akaar:~# xm info > host : akaar > release : 2.6.32-5-xen-686 > version : #1 SMP Mon Mar 26 10:32:49 UTC 2012 > machine : i686 > nr_cpus : 24 > nr_nodes : 2 > cores_per_socket : 6 > threads_per_core : 2 > cpu_mhz : 2666 > hw_caps : > bfebfbff:2c100800:00000000:00001f40:029ee3ff:00000000:00000001:00000000 > virt_caps : hvm hvm_directio > total_memory : 98291 > free_memory : 95010 > node_to_cpu : node0:0,2,4,6,8,10,12,14,16,18,20,22 > node1:1,3,5,7,9,11,13,15,17,19,21,23 > node_to_memory : node0:47110 > node1:47899 > node_to_dma32_mem : node0:0 > node1:2990 > max_node_id : 1 > xen_major : 4 > xen_minor : 0 > xen_extra : .1 > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 > hvm-3.0-x86_32p hvm-3.0-x86_64 > xen_scheduler : credit > xen_pagesize : 4096 > platform_params : virt_start=0xf9c00000 > xen_changeset : unavailable > xen_commandline : placeholder dom0_mem=1024M > cc_compiler : gcc version 4.4.5 (Debian 4.4.5-8) > cc_compile_by : waldi > cc_compile_domain : debian.org > cc_compile_date : Mon Nov 7 09:16:11 CET 2011 > xend_config_format : 4 > > root@akaar:~# xm list > Name ID Mem VCPUs State > Time(s) > Domain-0 0 997 24 r----- 1080.2 > guinan 2 4096 4 -b---- 0.0 > > root@akaar:~# xm vcpu-list > Name ID VCPU CPU State Time(s) CPU > Affinity > Domain-0 0 0 20 -b- 747.4 any cpu > Domain-0 0 1 17 -b- 80.2 any cpu > Domain-0 0 2 12 -b- 105.4 any cpu > Domain-0 0 3 19 -b- 49.0 any cpu > Domain-0 0 4 4 -b- 30.7 any cpu > Domain-0 0 5 13 -b- 35.9 any cpu > Domain-0 0 6 21 -b- 52.3 any cpu > Domain-0 0 7 23 -b- 37.9 any cpu > Domain-0 0 8 2 -b- 26.9 any cpu > Domain-0 0 9 7 -b- 32.7 any cpu > Domain-0 0 10 14 -b- 34.8 any cpu > Domain-0 0 11 16 -b- 26.2 any cpu > Domain-0 0 12 10 -b- 23.7 any cpu > Domain-0 0 13 8 -b- 38.7 any cpu > Domain-0 0 14 5 -b- 34.8 any cpu > Domain-0 0 15 11 -b- 30.3 any cpu > Domain-0 0 16 3 -b- 29.3 any cpu > Domain-0 0 17 18 -b- 41.2 any cpu > Domain-0 0 18 14 -b- 41.4 any cpu > Domain-0 0 19 1 -b- 45.6 any cpu > Domain-0 0 20 6 r-- 43.6 any cpu > Domain-0 0 21 15 -b- 29.2 any cpu > Domain-0 0 22 0 -b- 23.2 any cpu > Domain-0 0 23 7 -b- 30.5 any cpu > guinan 2 0 9 -b- 72.1 any cpu > guinan 2 1 15 -b- 2.9 any cpu > guinan 2 2 19 -b- 7.4 any cpu > guinan 2 3 13 -b- 6.0 any cpu > ========================================================================================================= > > > DomU > ========================================================================================================= > debian Squeeze 64Bit 1GB RAM / 4 vcpus / no pinning: > > root@guinan:~# uname -a > Linux guinan 2.6.32-5-xen-amd64 #1 SMP Thu Mar 22 21:14:26 UTC 2012 x86_64 > GNU/Linux > > root@guinan:~# free > total used free shared buffers cached > Mem: 1049180 227444 821736 0 20060 73712 > -/+ buffers/cache: 133672 915508 > Swap: 2101176 0 2101176 > > > > Attach Storage to DomU guinan > root@akaar:~# xm block-attach guinan phy:/dev/vg0/performance-test xvdc1 w > > lvcreate --name=performance-test --size=20G --stripes=6 vg0 > > Cheers > Ulli > -- > Ulrich Hochholdinger > Bereichsleiter Infrastruktur > > Netpioneer GmbH, Beiertheimer Allee 18, 76137 Karlsruhe > Geschäftsführer: Jürgen Diller > HRB: 107521 Handelsregister Mannheim > > Tel: 0721 / 9 20 60 90 > Fax: 0721 / 9 20 60 30 > E-Mail: Ulrich.Hochholdinger@xxxxxxxxxxxxx > www: http://www.netpioneer.de > > _______________________________________________ > Xen-users mailing list > Xen-users@xxxxxxxxxxxxx > http://lists.xen.org/xen-users Hi, I am guesing you are using Debian Squeeze. Pure PV domain performance on the Squeeze pvops dom0 kernel isn't too bad but definitely isn't on par with 3.2 pvops dom0 kernels. I would highly suggest not using dd for performance testing unless you write much more than 4GB. Atleast 2x the ammount of RAM should be written to ensure the Linux disk cache isn't playing with your data. Using highly compressible zeros isn't really that great either for managing disks that may be behind a raid controller etc. To be frank there is a huge amount to know about properly benchmarking I/O performance and detailing all of it goes beyond the depth of this post but atleast do the following: 1 ) Use the Flexible IO Tester also known as fio. It's by far the best and most reliable I/O testing tool. 2) Where possible use raw volumes, where not use LVM (You are using LVM which is good) 3) Always write atleast 2x the amount of RAM you have in the dom0, for best results write much more. For instance for a 4GB dom0 when I am doing SSD performance testing I write around 100GB of data. If you want really good I/O performance because it's important for your workload I would highly recommend using a more recent 3.2 dom0 kernel. I understand this is not practical for some that need to use "Enterprise" distros but that is the price you pay for using old kernels. Debian Squeeze with 2.6.32 is incredibly stable and I would recommend leaving it alone if that's what matters to you most. Joseph. -- Founder | Director | VP Research Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56 99 52 | Mobile: 0428 754 846 _______________________________________________ Xen-users mailing list Xen-users@xxxxxxxxxxxxx http://lists.xen.org/xen-users
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |