Re: [Xen-users] High Number of VMs

[John Madden <jmadden@xxxxxxxxxxx>]

On 09/09/2011 03:07 PM, Ciro Iriarte wrote:
> Hi, I'm curious about how you guys deal with big virtualization
> installations. To this date we only dealt with a small number of VM's
> (~10)on not too big hardware (2xquad xeons+16GB ram). As I'm the
> "storage guy" I find it quite convenient to present to the dom0s one
> LUN per VM that makes live migration possible but without the cluster
> file system or cLVM complexity. The thing is that Linux has a 255 SCSI
> device limit apparently (255/2= ~128 with multipath) and that won't
> scale in big installations (300 VMs for example).
>
> Any experiences on this scale?
I run a couple hundred VM's across a handful of blades.  I recommend 
going to fewer, larger LUNs, carving them up with LVM, and handing out 
LV's to your VM's.  You don't actually need cLVM to do this!  All the 
cluster infra does (for its nasty administrative overhead) is keep the 
LVM metadata (not your actual data) consistent through cluster-wide 
locks.  You can manage yourself by, for example, making changes on one 
node and refreshing the other nodes with things like 'vgscan -ay'.  I 
typically allocate LUNs 500GB at a time.  You can invent means of 
keeping things consistent that work for your environment, just test them 
first.
If you really want data security -- preventing one node from hosing data 
that's accessible from another node -- you'll have to go with a cluster 
filesystem.  That still doesn't help keep your multipathing and LVM 
configs consistent though, so I think you're better off just skipping 
that step.
John




--
John Madden / Sr UNIX Systems Engineer
Office of Technology / Ivy Tech Community College of Indiana

    Free Software is a matter of liberty, not price. To understand
    the concept, you should think of Free as in 'free speech,' not
    as in 'free beer.' -- Richard Stallman

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users