Re: [Xen-users] My future plan

[Bart Coninckx <bart.coninckx@xxxxxxxxxx>]

Hi Jonathan,

I use a DRBD bases IET install. It syncs between the nodes with two bonded 
Intel e1000 NICs. I use the same network cards to connect to the Xen 
hypervisors. 
MIND YOU: I use dual port NICs  (two in total on the storga servers) but I 
CROSS the connections: that is, I connect one port of one card to the Xen 
nodes, but I use the other for the DRBD sync; And the other way around of 
course. This way, if a card breaks, I still have things running. To be able to 
use two switches in between Xen hosts and the storage, I use multipathing to 
connect to the iSCSI LUNs. This results in higher speed and network 
redundancy. It would make no sense to use more than 2 ports since DRBD cannot 
sync faster, but also, as mentioned before, it seems that bonding more than 2 
does not result in higher speeds. 
This however is easily tested with netperf . I would be happy to hear someones 
testresults about this.

O yes, if you don't get the expected speeds with bonded cards in mode 0, try 
looking at tcp_reordering in /proc/sys/net/ipv4 something ...


On Wednesday 09 June 2010 14:53:28 Jonathan Tripathy wrote:
> So should I just go with 2 NICs for the storage server then?
> 
> In your future setup, how many NICs are you using for the storage server
>  and how many for the nodes? I take it you're using software iSCSI?
> 
> ________________________________
> 
> From: Bart Coninckx [mailto:bart.coninckx@xxxxxxxxxx]
> Sent: Wed 09/06/2010 11:25
> To: xen-users@xxxxxxxxxxxxxxxxxxx
> Cc: Jonathan Tripathy; Michael Schmidt
> Subject: Re: [Xen-users] My future plan
> 
> 
> 
> On the DRBD mailing lists I've seen a couple of times that they did tests
>  with bonding and they claim that a bond with more than 2 NICs will
>  actually decrease performance because of the TCP reordering that needs to
>  be done.
> 
> That's the reason why I limit the storage connection to two NICs. I have a
> very similar to yours in the making by the way.
> 
> On Tuesday 08 June 2010 15:55:47 Jonathan Tripathy wrote:
> > Hi Michael,
> >
> > Thanks for the tips using SSD for the node OS drives.
> >
> > Regarding the NIC, I was thinking about using this for the nodes:
> >
> > http://www.intel.com/products/server/adapters/pro1000pt-dualport/pro1000p
> >t- dualport-overview.htm
> >
> > and this for the server:
> >
> > http://www.intel.com/products/server/adapters/pro1000pt-quadport-low-prof
> >il e/pro1000pt-quadport-low-profile-overview.htm
> >
> > Are those the cards you were talking about? They are very cheap on ebay
> > you see...
> >
> > Think 4 port bonding for the server is good enough for 8 nodes?
> >
> > Thanks
> >
> > ________________________________
> >
> > From: Michael Schmidt [mailto:michael.schmidt@xxxxxxxxxx]
> > Sent: Tue 08/06/2010 14:49
> > To: Jonathan Tripathy; Xen-users@xxxxxxxxxxxxxxxxxxx
> > Subject: Re: [Xen-users] My future plan
> >
> >
> > Hi Jonathan,
> >
> > you should think about flash or SD cards as xen-boot-drive.
> > This provides you lower costs and higher energy efficiency.
> > If you mount /tmp and /var/log to an tmpfs, this disks works very well
> > and long.
> >
> > If you dont need so much disk space for your storage, use sas disks.
> > SAS (10k/15k) disks provides you many more IOPs than sata disks (more
> > IOPS per $/EUR as well). And very important: A very large cache for your
> > raid controller.
> >
> > Intel e1000e is a pretty good choice. This cards have a large buffer and
> >  generates just a few interrupts on your CPUs (in comparison to the
> >  Broadcom NICs).
> >
> > Best Regards
> >
> > Michael Schmidt
> > Am 08.06.10 14:55, schrieb Jonathan Tripathy:
> >
> >       My future plan currently looks like this for my VPS hosting
> > solution, so any feedback would be appreciated:
> >
> >       Each Node:
> >       Dell R210 Intel X3430 Quad Core 8GB RAM
> >       Intel PT 1Gbps Server Dual Port NIC using linux "bonding"
> >       Small pair of HDDs for OS (Probably in RAID1)
> >       Each node will run about 10 - 15 customer guests
> >
> >
> >       Storage Server:
> >       Some Intel Quad Core Chip
> >       2GB RAM (Maybe more?)
> >       LSI 8704EM2 RAID Controller (Think this controller does 3 Gbps)
> >       Battery backup for the above RAID controller
> >       4 X RAID10 Arrays (4 X 1.5TB disks per array, 16 disks in total)
> >       Each RAID10 array will connect to 2 nodes (8 nodes per storage
> > server) Intel PT 1Gbps Quad port NIC using Linux bonding
> >       Exposes 8 X 1.5GB iSCSI targets (each node will use one of these)
> >
> >       HP Procurve 1800-24G switch to create 1 X 4 port trunk (for storage
> >  server), and 8 X 2 port trunk (for the nodes)
> >
> >       What you think? Any tips?
> >
> >       Thanks
> >
> >
> >       _______________________________________________
> >       Xen-users mailing list
> >       Xen-users@xxxxxxxxxxxxxxxxxxx
> >       http://lists.xensource.com/xen-users
> 

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users