Re: [Xen-API] VMs are using more memory than reported and expected

[Alexandre Kouznetsov <alk@xxxxxxxxxx>]

Hello.

I have searched for xen memory overhead and found a few references of 
the issue, but no clear explanation. If you happen to know a good 
source, could you share a link?
As I understand it, each Dom is consuming the RAM it has been assigned 
plus some amount known as "memory-overhead", which size depend on Dom's 
assigned memory. I would like to know, what else does it depends on 
(total host's RAM?) and how to predict it.
When I do my calculations taking the memory-overhead into account, I get 
closer numbers but it still does not check. It seems like "xl list" does 
not take memory-overhead into account at all, while xentop gives a 
closer, but still inaccurate number in the header.
Used memory, calculating only assigned RAM: 29.72G
Used memory, calculating assigned RAM plus overhead: 30.26G
Used memory, as reported by xentop: 30.70G
I took memory-overhead values from "xe vm-param-list".

I'm still missing 1.25GB reported by xentop as "Free" (unusable if I try 
to assign it to a VM) and the difference between xentop's "used memory" 
vs sum of all assigned RAM and overheads.
On 
https://onapp.zendesk.com/entries/22086947-Optimize-Domain-0-and-Memory-Overhead-for-Xen-Hypervisors 
I have found a magic formula:
dom0_ram + (total_ram * .024) + 128 = memory_overhead
No reference where did it came from.

Thank you.

El 09/05/13 00:07, Brian Menges escribió:
> Depending on the VM booted, there is memory overhead:
>
> memory-overhead ( RO): 4194304
>
> Here's how I found my memory overhead for the Debian 6.0 (Squeeze) 64-bit image:
>
> [root@xs-01 ~]# xe template-list |grep -i 'squeeze' -B1
> uuid ( RO)                : b0f02f5d-d581-0e0a-8f85-40c3a65d660e
>            name-label ( RW): Debian Squeeze 6.0 (32-bit)
>      name-description ( RW): Template that allows VM installation from Xen-aware 
> Debian-based distros. To use this template from the CLI, install your VM using vm-install, 
> then set other-config-install-repository to the path to your network repository, e.g. 
> http://<server>/<path>
> In order to install Debian Squeeze from CD/DVD the multi-arch ISO image is 
> required.
> --
> uuid ( RO)                : 2122992d-d430-e380-625e-d6049cfae018
>            name-label ( RW): Debian Squeeze 6.0 (64-bit)
>      name-description ( RW): Template that allows VM installation from Xen-aware 
> Debian-based distros. To use this template from the CLI, install your VM using vm-install, 
> then set other-config-install-repository to the path to your network repository, e.g. 
> http://<server>/<path>
> In order to install Debian Squeeze from CD/DVD the multi-arch ISO image is 
> required.
> [root@xs-01 ~]# xe template-param-get uuid=2122992d-d430-e380-625e-d6049cfae018 
> param-name=memory-overhead
> 4194304
>
> As you can see, there's a few megs of overhead for just this VM, then there's 
> the ballooning range for dom0 that you have to consider, which I'm not 
> positive, but I think it may be 2G and you seem to have taken that into 
> account. The overhead range appears to be that last 1gb, and I'm sure other 
> things get crammed in there too like your ramdisk perhaps, xen netback buffers, 
> video card buffers, etc.
>
> - Brian Menges
> Principal Engineer, DevOps
> GoGrid | ServePath | ColoServe | UpStream Networks
>
>
> -----Original Message-----
> From: xen-api-bounces@xxxxxxxxxxxxx [mailto:xen-api-bounces@xxxxxxxxxxxxx] On 
> Behalf Of Alexandre Kouznetsov
> Sent: Wednesday, May 08, 2013 16:11
> To: xen-api@xxxxxxxxxxxxx
> Subject: [Xen-API] VMs are using more memory than reported and expected
>
> Hello.
>
> I have a host machine that has 32GB RAM and runs XCP 1.6. I expected to be able 
> to allocate up to 5 VM's using 6GB of RAM each, but XCP complains saying it 
> can't start the last VM because of RAM shortage. It looks like somebody is 
> eating more RAM that it should.
>
> My goal is to run 5 equal VM's on this host. As last resource, I may calculate 
> the appropriate RAM size to make them all fit (something around 5.8G), but this 
> is issue is really unexpected. I hoped to give 30GB to VMs and have up to 2GB 
> left for Dom0 and the roundings.
>
>
> I have inspected the RAM usage with "xl list" and "xentop" commands. As you can 
> see, I had to set one VM to use 5GB instead of 6GB of RAM.
> (hope the mail does not breaks the formatting too much)
>
> "xl list" output:
> Name                      ID   Mem VCPUs     State    Time(s)
> Domain-0                   0   741     4     r-----  1394433.5
> VM1                        1  6144     4     r-----  592272.6
> VM2                        2  6144     1     -b----   43681.0
> VM3                        3  6144     1     -b----    6635.9
> VM4                        4  6144     4     r-----   46929.5
> VM5                        5  5120     4     -b----   29723.3
>
> Xentop's output is rather large, but this are the numbers:
> Mem: 33507896k total, 32194356k used, 1313540k free
> NAME      MEM(k)
> Domain-0  759040
> VM1      6291456
> VM2      6291456
> VM3      6291456
> VM4      6291456
> VM5      5242880
>
> So, If I sum VM's memory consumption one by one (6 machines including Dom0), it gives me 
> 31167744, while xentop reports "32194356k used", almost 1GB is missing.
> xentop says I have "1313540k free", but if I set all 5 VMs to 6GB, the last one 
> can't start.
>
> Please advice, what am I missing?
> How came that the running domains have all less than 30GB assigned, but I'm 
> unable to assign the other 2GB?
> Why can't I at least allocate the "free" gigabyte that xentop reports?
>
> Thank you.
>
> --
> Alexandre Kouznetsov
>
>
> _______________________________________________
> Xen-api mailing list
> Xen-api@xxxxxxxxxxxxx
> http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api
>
> The information contained in this message, and any attachments, may contain 
> confidential and legally privileged material. It is solely for the use of the 
> person or entity to which it is addressed.  Any review, retransmission, 
> dissemination, or action taken in reliance upon this information by persons or 
> entities other than the intended recipient is prohibited. If you receive this 
> in error, please contact the sender and delete the material from any computer.
>
> _______________________________________________
> Xen-api mailing list
> Xen-api@xxxxxxxxxxxxx
> http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

--
Alexandre Kouznetsov


_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api