[Xen-users] Performance problems with domU on drbd

[Reto Gantenbein <reto.gantenbein@xxxxxxxxxxx>]

Hello everybody

After 3 days messing around I'm at my wits' end. I have two Sun X4200M2
servers which I would like to setup as Xen dom0s with a drbd-mirrored
file system for paravirtualized domU images.

I setup an LVM2 on top of DRBD to create the domU volumes and to be able
to backup the domUs with LVM snapshot.

I was able to configure this setup and make it run, but when accessing
the block-frontend device in domU this is terrible slow. E.g. copying a
5G file via scp and gigabit Ethernet onto the domU leads to a lot of
stalls during transfer. The peak transfer rate is about 5MB/s every 1-2
minutes but generally it's about a few hundred KB/s. When copying the
same file onto the DRBD volume mounted directly in dom0 it works with
the expected 30-40MB/s. Another test was to sync the portage tree but
also this takes maybe about 20 minutes or spontaneously raises some
timeout errors. Sometimes after a while the disk fully detaches from the
domU, so every call of a binary in domU leads to a "Input/output error".
I guess the domU kernel adds this troubles, but why? There are no
statements in any log files. E.g. /var/log/xen/xend-debug.log doesn't
notify the detached block device.

I use Gentoo Linux and tried so far with xen-3.1.2 and xen-3.2.0,
xen-sources-2.6.18-r8, xen-sources-2.6.20-r6 and xen-sources-2.6.22,
drbd-8.0.7, drbd-8.0.8 and drbd-8.0.11.

The longer the more I expect a general problem with Xen and this setup,
a kernel problem or a misconfiguration. Attached are the most important
configurations for this setup.

Are there any hints or other experiences for such a setup? All ideas are
welcome.

Thanks for reading and giving any hints...
Cheers Reto Gantenbein


################ Configurations #################################

/etc/drbd.conf
,----------------------------------------------------------------
| global {
|     usage-count no;
|     dialog-refresh 1;
| }
|
| common {
|     startup {
|         wfc-timeout 60;
|         degr-wfc-timeout 60;
|     }
|
|     disk {
|         on-io-error detach;
|         # also tried 'pass-on' with same behavior
|     }
|
|     syncer {
|         rate 500M;
|         al-extents 80;
|     }
| }
|
| resource "gentoo" {
|     protocol C;
|     net {
|         # required for xen live migration
|         allow-two-primaries;
|         after-sb-0pri discard-zero-changes;
|         after-sb-1pri discard-secondary;
|    }
|
|    on vserver03 {
|         device        /dev/drbd1;
|         disk          /dev/xen-vg/gentoo;
|         address       10.0.0.1:7789;
|         meta-disk     internal;
|    }
|
|    on vserver04 {
|        device         /dev/drbd1;
|        disk           /dev/xen-vg/gentoo;
|        address        10.0.0.2:7789;
|        meta-disk      internal;
|    }
| }
`----------------------------------------------------------------

/etc/xen/gentoo
,----------------------------------------------------------------
| name = "gentoo"
|
| kernel = "/cluster/xen/kernel/vmlinuz-gentoo-domU"
| extra = ""
| root = "/dev/hda1"
|
| memory = 4096
| vcpus = 3
| disk = [ 'drbd:gentoo,hda1,w' ]
| vif = [ 'mac=00:40:a6:00:45:01,bridge=xenbr-core',
|         'mac=00:40:a6:00:45:02,bridge=xenbr-public' ]
`----------------------------------------------------------------

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users