|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] RE: [Xen-devel] Re: [Xen-users] Max. PV and HVM Guests
Hmmm... I'll bet the problem with dom0 crashing is that dom0 is pv_ops (2.6.31-based) and this patch, which has been in 2.6.18.8-based dom0 for some time, has never been put into pv_ops: http://lists.xensource.com/archives/html/xen-devel/2008-04/msg00143.html That would explain why some people are seeing this problem and others are not, and why setting dom0_mem seems to solve the problem (as dom0_mem effectively shuts off ballooning in dom0). Nick and others, if you are interested in detail on ballooning and improving memory utilization within and between guests, see: http://wiki.xensource.com/xenwiki/Open_Topics_For_Discussion?action=AttachFile&do=get&target=Memory+Overcommit.pdf and http://oss.oracle.com/projects/tmem (Tmem is in xen-unstable and in Oracle VM 2.2... it takes a few steps to set it up.) > -----Original Message----- > From: Pasi Kärkkäinen [mailto:pasik@xxxxxx] > Sent: Monday, November 09, 2009 8:18 AM > To: Nick Couchman > Cc: xen-devel@xxxxxxxxxxxxxxxxxxx; Mr. Teo En Ming (Zhang Enming); > xen-users@xxxxxxxxxxxxxxxxxxx; Robert Dunkley > Subject: Re: [Xen-devel] Re: [Xen-users] Max. PV and HVM Guests > > > On Mon, Nov 09, 2009 at 08:06:54AM -0700, Nick Couchman wrote: > > > > > > >>> On 2009/11/09 at 05:05, Pasi Kärkkäinen<pasik@xxxxxx> wrote: > > > On Mon, Nov 09, 2009 at 08:01:00PM +0800, Mr. Teo En Ming (Zhang > > Enming) > > > wrote: > > >> No, I didn't limit dom0 memory in grub.conf. > > >> > > > > > > You should. > > > > Really? I thought current conventional wisdom was to allow Xen to > > self-manage memory in both dom0 and domUs, and not to > manually adjust > > this? I run several Xen systems with anywhere from 8 to 24 > GB of RAM > > and 20 to 30 domUs on some of these systems and have > *never* specified > > the dom0 memory at boot time - the Xen ballooning has > always functioned > > perfectly fine, and never crashed my dom0. > > > > Yes, Xen is totally OK with this, but dom0 Linux has more problems.. > > > Furthermore, while I'm not > > Linux developer and so not familiar with how Linux > calculates buffering > > and caching, I do know that my Linux systems dynamically > manage buffers > > and caches, and when memory is reduced or some application > requires a > > larger amount of physical memory, Linux reduces the amount > of data in > > buffers and caches. > > > > Yeah, it has to do with sizing the network buffers, caches etc.. > > It shouldn't _crash_, so Teo is seeing some bug I believe. But it has > always been "best practice" to limit dom0 memory - and prevent weird > things happening later (like "memory squeeze in netback driver"). > > > Of course, a lot of this depends on what you're doing in > dom0 - on my > > Xen servers, my dom0 is strictly for Xen management - I'm > not running > > anything else in dom0 that would require large amounts of > memory, memory > > buffers and caches, etc. > > > > Teo is running graphical stuff, X etc so it's a bit different.. > > -- Pasi > > > > > > > > > If dom0 has all the memory at boot time, you need to balloon down > > dom0 > > > memory every time you create a new guest - this can (and > will) cause > > > > > problems with the dom0 linux kernel. > > > > > > Linux calculates some internal parameters/buffers/values based on > > the > > > _boot time_ amount of memory. And when the amount of > memory goes down > > to > > > only a small fraction of that while creating new guests bad things > > can > > > happen.. > > > > > > It still shouldn't crash though.. I bet your problem will > get fixed > > when > > > you limit the dom0 memory to say dom0_mem=512M and reboot. > > > > > > -- Pasi > > > > > >> Here's my xm info output after I have shutdown all the virtual > > machines. > > >> > > >> [root@fedora11-x86-64-host ~]# xm list > > >> Name ID Mem > VCPUs > > State > > >> Time(s) > > >> Domain-0 0 2812 2 > > r----- > > >> 3242.5 > > >> [root@fedora11-x86-64-host ~]# xm info > > >> host : fedora11-x86-64-host > > >> release : 2.6.30-rc3-enming.teo-tip > > >> version : #1 SMP Wed Aug 19 23:14:15 SGT 2009 > > >> machine : x86_64 > > >> nr_cpus : 2 > > >> nr_nodes : 1 > > >> cores_per_socket : 2 > > >> threads_per_core : 1 > > >> cpu_mhz : 2800 > > >> hw_caps : > > >> > > > bfebfbff:20100800:00000000:00000140:0400e3bd:00000000:00000001 > :00000000 > > >> virt_caps : hvm hvm_directio > > >> total_memory : 6039 > > >> free_memory : 3124 > > >> node_to_cpu : node0:0-1 > > >> node_to_memory : node0:3124 > > >> xen_major : 3 > > >> xen_minor : 5 > > >> xen_extra : -unstable > > >> xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p > > hvm-3.0-x86_32 > > >> hvm-3.0-x86_32p hvm-3.0-x86_64 > > >> xen_scheduler : credit > > >> xen_pagesize : 4096 > > >> platform_params : virt_start=0xffff800000000000 > > >> xen_changeset : Tue Sep 01 11:34:31 2009 +0100 > > > 20143:a7de5bd776ca > > >> xen_commandline : iommu=1 > > >> cc_compiler : gcc version 4.4.1 20090725 (Red Hat > > 4.4.1-2) > > >> (GCC) > > >> cc_compile_by : root > > >> cc_compile_domain : (none) > > >> cc_compile_date : Thu Sep 10 07:01:13 SGT 2009 > > >> xend_config_format : 4 > > >> > > >> -- > > >> Mr. Teo En Ming (Zhang Enming) Dip(Mechatronics) > > BEng(Hons)(Mechanical > > >> Engineering) > > >> Alma Maters: > > >> (1) Singapore Polytechnic > > >> (2) National University of Singapore > > >> My Primary Blog: > > [1]http://teo-en-ming-aka-zhang-enming.blogspot.com > > >> My Secondary Blog: [2]http://enmingteo.wordpress.com > > >> My Youtube videos: [3]http://www.youtube.com/user/enmingteo > > >> Email: [4]space.time.universe@xxxxxxxxx > > >> Mobile Phone (Starhub Prepaid): +65-8369-2618 > > >> Street: Bedok Reservoir Road > > >> Country: Singapore > > >> > > >> On Mon, Nov 9, 2009 at 7:54 PM, Pasi Kärkkäinen > <[5]pasik@xxxxxx> > > wrote: > > >> > > >> On Mon, Nov 09, 2009 at 06:52:37PM +0800, Mr. Teo En Ming > > (Zhang > > > Enming) > > >> wrote: > > >> > Hi, > > >> > > > >> > Please watch this 4-minute video at > > >> > [1][6]http://www.youtube.com/watch?v=LbLaPpwNAx4 > > >> > > > >> > I have only started 3 HVM Linux guests with 1 > GB ram each. > > I can't > > >> start > > >> > the 4th HVM guest. If I attempt to start the > 4th instance, > > it will > > >> crash > > >> > dom0. > > >> > > > >> > Are there anything in the xm dmesg output that could > > explain the > > >> low limit > > >> > to the number of VMs that I could start before dom0 > > becomes > > >> unresponsive? > > >> > > > >> > > >> Have you limited dom0 memory (by specifying > dom0_mem=XMB option > > in > > >> grub.conf for xen.gz) ? > > >> > > >> What does "xm info" say about free memory before > starting any > > guests? > > >> -- Pasi > > >> > > >> References > > >> > > >> Visible links > > >> 1. http://teo-en-ming-aka-zhang-enming.blogspot.com/ > > >> 2. http://enmingteo.wordpress.com/ > > >> 3. http://www.youtube.com/user/enmingteo > > >> 4. mailto:space.time.universe@xxxxxxxxx > > >> 5. mailto:pasik@xxxxxx > > >> 6. http://www.youtube.com/watch?v=LbLaPpwNAx4 > > > > > > > > -------- > > This e-mail may contain confidential and privileged > material for the sole use of the intended recipient. If this > email is not intended for you, or you are not responsible for > the delivery of this message to the intended recipient, > please note that this message may contain SEAKR Engineering > (SEAKR) Privileged/Proprietary Information. In such a case, > you are strictly prohibited from downloading, photocopying, > distributing or otherwise using this message, its contents or > attachments in any way. If you have received this message in > error, please notify us immediately by replying to this > e-mail and delete the message from your mailbox. Information > contained in this message that does not relate to the > business of SEAKR is neither endorsed by nor attributable to SEAKR. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@xxxxxxxxxxxxxxxxxxx > http://lists.xensource.com/xen-devel > _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |