Re: [Xen-API] VMs are using more memory than reported and expected

[Brian Menges <bmenges@xxxxxxxxxx>]

Pretty sure that dom0 is allowed to balloon up to 2gb, of which usually ~750m 
is used unless the hypervisor is tasking hard.

This 'ballooning memory' won't be accounted for in xl list, but is still not 
available for other VMs incase it is needed.

- Brian Menges
Principal Engineer, DevOps
GoGrid | ServePath | ColoServe | UpStream Networks


-----Original Message-----
From: xen-api-bounces@xxxxxxxxxxxxx [mailto:xen-api-bounces@xxxxxxxxxxxxx] On 
Behalf Of Alexandre Kouznetsov
Sent: Thursday, May 09, 2013 10:14
To: xen-api@xxxxxxxxxxxxx
Subject: Re: [Xen-API] VMs are using more memory than reported and expected

Hello.

I have searched for xen memory overhead and found a few references of the 
issue, but no clear explanation. If you happen to know a good source, could you 
share a link?

As I understand it, each Dom is consuming the RAM it has been assigned plus 
some amount known as "memory-overhead", which size depend on Dom's assigned 
memory. I would like to know, what else does it depends on (total host's RAM?) 
and how to predict it.

When I do my calculations taking the memory-overhead into account, I get closer 
numbers but it still does not check. It seems like "xl list" does not take 
memory-overhead into account at all, while xentop gives a closer, but still 
inaccurate number in the header.
Used memory, calculating only assigned RAM: 29.72G Used memory, calculating 
assigned RAM plus overhead: 30.26G Used memory, as reported by xentop: 30.70G I 
took memory-overhead values from "xe vm-param-list".

I'm still missing 1.25GB reported by xentop as "Free" (unusable if I try to 
assign it to a VM) and the difference between xentop's "used memory"
vs sum of all assigned RAM and overheads.

On
https://onapp.zendesk.com/entries/22086947-Optimize-Domain-0-and-Memory-Overhead-for-Xen-Hypervisors
I have found a magic formula:
dom0_ram + (total_ram * .024) + 128 = memory_overhead No reference where did it 
came from.

Thank you.

El 09/05/13 00:07, Brian Menges escribió:
> Depending on the VM booted, there is memory overhead:
>
> memory-overhead ( RO): 4194304
>
> Here's how I found my memory overhead for the Debian 6.0 (Squeeze) 64-bit 
> image:
>
> [root@xs-01 ~]# xe template-list |grep -i 'squeeze' -B1
> uuid ( RO)                : b0f02f5d-d581-0e0a-8f85-40c3a65d660e
>            name-label ( RW): Debian Squeeze 6.0 (32-bit)
>      name-description ( RW): Template that allows VM installation from
> Xen-aware Debian-based distros. To use this template from the CLI, install 
> your VM using vm-install, then set other-config-install-repository to the 
> path to your network repository, e.g. http://<server>/<path> In order to 
> install Debian Squeeze from CD/DVD the multi-arch ISO image is required.
> --
> uuid ( RO)                : 2122992d-d430-e380-625e-d6049cfae018
>            name-label ( RW): Debian Squeeze 6.0 (64-bit)
>      name-description ( RW): Template that allows VM installation from
> Xen-aware Debian-based distros. To use this template from the CLI, install 
> your VM using vm-install, then set other-config-install-repository to the 
> path to your network repository, e.g. http://<server>/<path> In order to 
> install Debian Squeeze from CD/DVD the multi-arch ISO image is required.
> [root@xs-01 ~]# xe template-param-get
> uuid=2122992d-d430-e380-625e-d6049cfae018 param-name=memory-overhead
> 4194304
>
> As you can see, there's a few megs of overhead for just this VM, then there's 
> the ballooning range for dom0 that you have to consider, which I'm not 
> positive, but I think it may be 2G and you seem to have taken that into 
> account. The overhead range appears to be that last 1gb, and I'm sure other 
> things get crammed in there too like your ramdisk perhaps, xen netback 
> buffers, video card buffers, etc.
>
> - Brian Menges
> Principal Engineer, DevOps
> GoGrid | ServePath | ColoServe | UpStream Networks
>
>
> -----Original Message-----
> From: xen-api-bounces@xxxxxxxxxxxxx
> [mailto:xen-api-bounces@xxxxxxxxxxxxx] On Behalf Of Alexandre
> Kouznetsov
> Sent: Wednesday, May 08, 2013 16:11
> To: xen-api@xxxxxxxxxxxxx
> Subject: [Xen-API] VMs are using more memory than reported and
> expected
>
> Hello.
>
> I have a host machine that has 32GB RAM and runs XCP 1.6. I expected to be 
> able to allocate up to 5 VM's using 6GB of RAM each, but XCP complains saying 
> it can't start the last VM because of RAM shortage. It looks like somebody is 
> eating more RAM that it should.
>
> My goal is to run 5 equal VM's on this host. As last resource, I may 
> calculate the appropriate RAM size to make them all fit (something around 
> 5.8G), but this is issue is really unexpected. I hoped to give 30GB to VMs 
> and have up to 2GB left for Dom0 and the roundings.
>
>
> I have inspected the RAM usage with "xl list" and "xentop" commands. As you 
> can see, I had to set one VM to use 5GB instead of 6GB of RAM.
> (hope the mail does not breaks the formatting too much)
>
> "xl list" output:
> Name                      ID   Mem VCPUs     State    Time(s)
> Domain-0                   0   741     4     r-----  1394433.5
> VM1                        1  6144     4     r-----  592272.6
> VM2                        2  6144     1     -b----   43681.0
> VM3                        3  6144     1     -b----    6635.9
> VM4                        4  6144     4     r-----   46929.5
> VM5                        5  5120     4     -b----   29723.3
>
> Xentop's output is rather large, but this are the numbers:
> Mem: 33507896k total, 32194356k used, 1313540k free
> NAME      MEM(k)
> Domain-0  759040
> VM1      6291456
> VM2      6291456
> VM3      6291456
> VM4      6291456
> VM5      5242880
>
> So, If I sum VM's memory consumption one by one (6 machines including Dom0), 
> it gives me 31167744, while xentop reports "32194356k used", almost 1GB is 
> missing.
> xentop says I have "1313540k free", but if I set all 5 VMs to 6GB, the last 
> one can't start.
>
> Please advice, what am I missing?
> How came that the running domains have all less than 30GB assigned, but I'm 
> unable to assign the other 2GB?
> Why can't I at least allocate the "free" gigabyte that xentop reports?
>
> Thank you.
>
> --
> Alexandre Kouznetsov
>
>
> _______________________________________________
> Xen-api mailing list
> Xen-api@xxxxxxxxxxxxx
> http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
>
> The information contained in this message, and any attachments, may contain 
> confidential and legally privileged material. It is solely for the use of the 
> person or entity to which it is addressed.  Any review, retransmission, 
> dissemination, or action taken in reliance upon this information by persons 
> or entities other than the intended recipient is prohibited. If you receive 
> this in error, please contact the sender and delete the material from any 
> computer.
>
> _______________________________________________
> Xen-api mailing list
> Xen-api@xxxxxxxxxxxxx
> http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
>


--
Alexandre Kouznetsov


_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

The information contained in this message, and any attachments, may contain 
confidential and legally privileged material. It is solely for the use of the 
person or entity to which it is addressed.  Any review, retransmission, 
dissemination, or action taken in reliance upon this information by persons or 
entities other than the intended recipient is prohibited. If you receive this 
in error, please contact the sender and delete the material from any computer.

_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api