[Xen-users] Slow disk performance in guests

[Roel van Meer <rolek@xxxxxxxxxx>]

Hi list,

I'm using a Compaq Proliant DL385G2, with two dual-core HE Opteron CPU's, 
16G memory and a bunch of 146G SAS disks in a hardware raid configuration. 
The machine is running SLES10SP1.
I created some LVM partitions to use in my Xen guests. When I mount these 
partitions in domU, performance is quite good. When I use them in a guest, 
however, write performance is really poor. I tried ReiserFS, ext3 and ex3 
mounted as ext2, but that really doesn't make any difference.
Below are two typical results of a run with bonnie with a 2G file. All 
values are in MB/s.
On domU:
Char write:   58
Rewrite:     239
Block write: 711
Char read:    68
Block read: 1283

On Guest:
Char write:   31
Rewrite:       7
Block write:  38
Char read:    65
Block read:  903

I ran these tests multiple times and these values are reproducable. The 
difference in write performance is obvious.
Could anyone shed some light ons this, or provide me with some hints on 
where to look? Below are the vm config file and dmesg output from the guest.

Thanks in advance,

roel

---/---
vm config file

disk = [ 'phy:/dev/system/vm1system,hda,w', 'phy:/dev/cciss/c0d0p5,hdb,w', 
'phy:/dev/system/vm1usrsap,hdc,w', 'phy:/dev/system/vm1sapmnt,hdd,w', 
'phy:/dev/system/vm1sapdata,hde,w', 'phy:/dev/cciss/c0d0p6,hdf,w', 
'phy:/dev/system/vm1temp,hdg,w', 'phy:/dev/system/test,hdh,w', ]
memory = 4096
vcpus = 1
cpus = "2"
builder = 'linux'
name = 'vm1'
vif = [ 'mac=00:16:3e:95:1d:66' ]
localtime = 1
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
extra = ' TERM=xterm'
bootloader = '/usr/lib/xen/boot/domUloader.py'
bootentry = 'hda:/boot/vmlinuz-xen,/boot/initrd-xen'


---/---
dmesg from guest

Bootdata ok (command line is  TERM=xterm)
Linux version 2.6.16.46-0.14-xen (geeko@buildhost) (gcc version 4.1.2 
20070115 (prerelease) (SUSE Linux)) #1 SMP Thu May 17 14:00:09 UTC 2007
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 0000000100800000 (usable)
On node 0 totalpages: 1050624
 DMA zone: 1050624 pages, LIFO batch:31
 DMA32 zone: 0 pages, LIFO batch:0
 Normal zone: 0 pages, LIFO batch:0
 HighMem zone: 0 pages, LIFO batch:0
No mptable found.
Built 1 zonelists
Kernel command line:  TERM=xterm
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 131072 bytes)
Xen reported: 2400.084 MHz processor.
Console: colour dummy device 80x25
Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
Software IO TLB disabled
Memory: 4096256k/4202496k available (1902k kernel code, 97476k reserved, 
798k data, 160k init)
Calibrating delay using timer specific routine.. 6001.89 BogoMIPS 
(lpj=12003788)
Security Framework v1.0.0 initialized
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
checking if image is initramfs... it is
Freeing initrd memory: 7124k freed
Brought up 1 CPUs
migration_cost=0
DMI not present or invalid.
Grant table initialized
NET: Registered protocol family 16
Brought up 1 CPUs
PCI: setting up Xen PCI frontend stub
ACPI: Subsystem revision 20060127
ACPI: Interpreter disabled.
xen_mem: Initialising balloon driver.
Setting mem allocation to 4194304 kiB
PCI: System does not support PCI
PCI: System does not support PCI
IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $
audit: initializing netlink socket (disabled)
audit(1183375183.864:1): initialized
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
rtc: IRQ 8 is not free.
Non-volatile memory driver v1.2
i8042.c: No controller found.
RAMDISK driver initialized: 16 RAM disks of 128000K size 1024 blocksize
xencons_init: Initializing xen vfb; pass xencons=tty to prevent this
Xen virtual console successfully installed as xvc0
Event-channel device installed.
mice: PS/2 mouse device common for all mice
input: PC Speaker as /class/input/input0
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
NET: Registered protocol family 2
IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
NET: Registered protocol family 1
XENBUS: Device with no driver: device/vbd/768
XENBUS: Device with no driver: device/vbd/832
XENBUS: Device with no driver: device/vbd/5632
XENBUS: Device with no driver: device/vbd/5696
XENBUS: Device with no driver: device/vbd/8448
XENBUS: Device with no driver: device/vbd/8512
XENBUS: Device with no driver: device/vbd/8704
XENBUS: Device with no driver: device/vbd/8768
XENBUS: Device with no driver: device/vif/0
XENBUS: Device with no driver: device/console/0
netfront: Initialising virtual ethernet driver.
netfront: device eth0 has copying receive path.
Registering block device major 3
blkfront: hda: barriers enabled
hda: unknown partition table
blkfront: hdb: barriers enabled
hdb: unknown partition table
Registering block device major 22
blkfront: hdc: barriers enabled
hdc: unknown partition table
blkfront: hdd: barriers enabled
hdd: unknown partition table
Registering block device major 33
blkfront: hde: barriers enabled
hde: unknown partition table
blkfront: hdf: barriers enabled
hdf: unknown partition table
Registering block device major 34
blkfront: hdg: barriers enabled
hdg: unknown partition table
blkfront: hdh: barriers enabled
hdh: unknown partition table
ReiserFS: hda: found reiserfs format "3.6" with standard journal
ReiserFS: hda: using ordered data mode
reiserfs: using flush barriers
ReiserFS: hda: journal params: device hda, size 8192, journal first block 
18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
ReiserFS: hda: checking transaction log (hda)
blkfront: hda: write barrier op failed
blkfront: hda: barriers disabled
reiserfs: disabling flush barriers on hda
ReiserFS: hda: Using r5 hash to sort names
Adding 31254416k swap on /dev/hdb.  Priority:-1 extents:1 across:31254416k
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
device-mapper: 4.7.0-ioctl (2006-06-24) initialised: dm-devel@xxxxxxxxxx
dm-netlink version 0.0.2 loaded
loop: loaded (max 64 devices)
ReiserFS: hdc: found reiserfs format "3.6" with standard journal
ReiserFS: hdc: using ordered data mode
reiserfs: using flush barriers
ReiserFS: hdc: journal params: device hdc, size 8192, journal first block 
18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
ReiserFS: hdc: checking transaction log (hdc)
blkfront: hdc: write barrier op failed
blkfront: hdc: barriers disabled
reiserfs: disabling flush barriers on hdc
ReiserFS: hdc: Using r5 hash to sort names
[...]
AppArmor: AppArmor (version 2.0-19.43r6320) initialized
audit(1183375207.877:2): AppArmor (version 2.0-19.43r6320) initialized

audit(1183375220.582:3): audit_pid=2108 old=0 by auid=4294967295
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
IPv6 over IPv4 tunneling driver
8250: module not supported by Novell, setting U taint flag.
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
eth0: no IPv6 routers present
 

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users