rsize and wsize are the most
important ones to reduce your iowait. Those can dramatically change
performance.
The theory is that the best
performance is reached when you match the geometry of the actual disks to the
rsize/wsize nfs parameter.
Use dd to emulate different block
sizes and check the best performance for your environment.
Emerson
-----Original
Message-----
From: Geoff Galitz
[mailto:geoff@xxxxxxxxxx]
Sent: Tuesday, September 14, 2010 11:33
AM
To: Ribeiro Emerson Gomes;
xen-users@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-users] IOwaits over
NFS
We`ve used various NFS parameters
in our testing to resolve this. Here are three different examples of
what we have used on a single machine, with no significant
improvement:
1)
defaults,auto,tcp,bg,hard,intr,noatime 0 0
2)
auto,noatime,rsize=32768,wsize=32768,bg,nfsvers=3,intr,tcp,retrans=0 0
0
3)
noatime,bg,intr,nolock,tcp 0
0
Sent:
Tuesday, September 14,
2010 4:20
PM
Subject: RE:
[Xen-users] IOwaits over NFS
Please inform the
parameters you use in fstab when you mount your NFS share.
Something like this:
<server>:/<share>
<mountpoint> nfs rsize=8192, wsize=8192, timeo=14, intr, tcp, bg, rw, bg,
hard, nointr, rsize=32768, wsize=32768, tcp, actimeo=0,
vers=3,timeo=600
Emerson
-----Original
Message-----
From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-users-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Geoff Galitz
Sent: Tuesday,
September 14, 2010 11:04
AM
To:
xen-users@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-users] IOwaits over
NFS
Hello.
We have a
number of Xen 3.4.2. boxes which have constant iowaits at around
10% with
spikes up to 100% when accessing data over NFS. We have been
unable to
nail down the issue. Any advice?
System
info:
release
:
2.6.18-194.3.1.el5xen
version
: #1 SMP Thu May 13 13:49:53 EDT
2010
machine
:
x86_64
nr_cpus
:
16
nr_nodes
: 1
cores_per_socket :
4
threads_per_core :
2
cpu_mhz
:
2261
hw_caps
:
bfebfbff:28100800:00000000:00000340:009ce3bd:00000000:00000001:00000000
virt_caps
:
hvm
total_memory
:
32758
free_memory
:
3974
node_to_cpu
: node0:0-15
node_to_memory :
node0:3974
xen_major
:
3
xen_minor
:
4
xen_extra
:
.2
xen_caps
: xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p
hvm-3.0-x86_64
xen_scheduler
:
credit
xen_pagesize
: 4096
platform_params :
virt_start=0xffff800000000000
xen_changeset
:
unavailable
cc_compiler
: gcc version 4.1.2 20080704 (Red Hat
4.1.2-44)
cc_compile_by
: root
cc_compile_domain :
gitco.tld
cc_compile_date : Wed Nov
11 21:16:28 CET 2009
xend_config_format :
4
And lsmod from a
VM:
Module
Size Used
by
nfsd
284041
17
exportfs
38849 1
nfsd
auth_rpcgss
81889 1
nfsd
ipv6
424737
24
xfrm_nalgo
43333 1
ipv6
crypto_api
42945 1
xfrm_nalgo
i2c_dev
43849
0
i2c_core
56129 1
i2c_dev
nfs
289549
1
lockd
99185 3
nfsd,nfs
fscache
52385 1
nfs
nfs_acl
36673 2
nfsd,nfs
sunrpc
197513 18
nfsd,auth_rpcgss,nfs,lockd,nfs_acl
iptable_filter
36161
1
ip_tables
55329 1
iptable_filter
x_tables
50377 1
ip_tables
dm_mirror
54345
0
dm_multipath
55385
0
scsi_dh
41665 1
dm_multipath
scsi_mod
197337 1
scsi_dh
parport_pc
62313
0
lp
47121
0
parport
73293 2
parport_pc,lp
pcspkr
36289
0
xennet
61129 0
[permanent]
dm_raid45
98897
0
dm_message
36161 1
dm_raid45
dm_region_hash
46401 1
dm_raid45
dm_log
44993 3
dm_mirror,dm_raid45,dm_region_hash
dm_mod
100753 4
dm_mirror,dm_multipath,dm_raid45,dm_log
dm_mem_cache
39617 1
dm_raid45
ext3
168529
1
jbd
94257 1
ext3
uhci_hcd
57561
0
ohci_hcd
56053
0
ehci_hcd
65869
0
xenblk
50729 1
