Re: [Xen-devel] kernel log flooded with: xen_balloon: reserve_additional_memory: add_memory() failed: -17

[James Dingwall <james-xen@xxxxxxxxxxxxxx>]

On 2012-12-19 08:47, James Dingwall wrote:
> Hi,
>
> I have encountered an apparently benign error on two systems where
> the dom0 kernel log is flooded with messages like:
>
> [52482.163855] System RAM resource [mem 0x1b8000000-0x1bfffffff] 
> cannot be added
> [52482.163860] xen_balloon: reserve_additional_memory: add_memory() 
> failed: -17
> The first line is from drivers/xen/xen-balloon.c, the second from
> mm/memory_hotplug.c
>
> The trigger for the messages seems to be the first occasion that a
> Xen guest is shutdown.  I have noted this in a vanilla 3.6.7 and
> kernel 3.5.0-18 built from Ubuntu sources.  Xen version is 4.2.0.  It
> is not clear why the dom0 is kernel is trying to balloon up as the 
> Xen
> command line is specifies a fixed dom0 memory allocation and
> noselfballooning is specified for the kernel and ballooning is also
> disabled in the xend-config.sxp / xl.conf (one system using xm,
> another xl)
>
> xen command line:
> placeholder xsave=0 iommu=0 console=vga,com2 com2=115200,8n1 
> dom0_mem=max:6144m
> kernel command line:
> root=/dev/loop0 ro console=tty1 console=hvc0 earlyprintk=xen
> nomodeset noselfballooning
>
> Examining /proc/iomem does show that the dom0 memory allocation is
> actually 64kb short of 6144Mb:
>
> cat /proc/iomem | grep System\ RAM
> 00010000-0009bfff : System RAM      [573440 bytes]
> 00100000-cb2dffff : System RAM      [3407740928 bytes]
> 100000000-1b4d83fff : System RAM    [3034071040 bytes]
>
> Total system ram: 6442385408 - 6x2^30 = 65536
>
> The memory range indicated in the log message is "Unusable memory" in
> /proc/iomem:
> 1b4d84000-82fffffff : Unusable memory
>
> Another point of interest is that we have multiple "identical"
> hardware platforms (Dell T320) for the system running the 3.5.0-18
> kernel but only see this error on a slightly more recent system.
> Older systems show in /proc/iomem that all memory is System RAM.
>
> 100000000-82fffffff : System RAM  [older system BIOS 1.0]
>
> 100000000-1b4d83fff : System RAM  [newer system BIOS 1.3]
> 1b4d84000-82fffffff : Unusable memory
>
> The BIOS revision between the old and new has changed so I was
> wondering if it is possible that there is a white list which affects
> the impact of the kernel option:
> CONFIG_X86_RESERVE_LOW=64
> This is only a guess since the amount of memory reserved is
> equivalent to the short fall calculated above.  If this is the right
> area perhaps the dom0 calculation for its memory entitlement needs to
> be taught to not to try and hotplug the missing 64k when it has been
> reserved.
>
> If any other information would be useful then please let me know.
With some further investigation we have determined that the different 
BIOS version does not seem to be a factor and the key point is actually 
the Xen command line.  The reason that we had max: specified is that 
without it we could not boot the kernel/xen combination on an AMD 
platform.  I will do some further testing to see what the result of 
dom0_mem=6144m,max:6144m as suggested 
http://wiki.xen.org/wiki/Xen_Best_Practices gets us.
placeholder xsave=0 iommu=0 console=vga,com2 com2=115200,8n1 
dom0_mem=max:6144m
results in /proc/iomem having an unusable range and top reports 
6083900k of memory in dom0
placeholder xsave=0 iommu=0 console=vga,com2 com2=115200,8n1 
dom0_mem=6144m
no unusable range, top reports 5605976k of memory in dom0 and no log 
messages

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel