Re: [PATCH] x86/pod: Do not fragment PoD memory allocations

[George Dunlap <George.Dunlap@xxxxxxxxxx>]

> On Jan 28, 2021, at 10:42 PM, George Dunlap <george.dunlap@xxxxxxxxxx> wrote:
> 
> 
> 
>> On Jan 28, 2021, at 6:26 PM, Elliott Mitchell <ehem+xen@xxxxxxx> wrote:
>> 
>> On Thu, Jan 28, 2021 at 11:19:41AM +0100, Jan Beulich wrote:
>>> On 27.01.2021 23:28, Elliott Mitchell wrote:
>>>> On Wed, Jan 27, 2021 at 09:03:32PM +0000, Andrew Cooper wrote:
>>>>> So.?? What *should* happen is that if QEMU/OVMF dirties more memory than
>>>>> exists in the PoD cache, the domain gets terminated.
>>>>> 
>>>>> Irrespective, Xen/dom0 dying isn't an expected consequence of any normal
>>>>> action like this.
>>>>> 
>>>>> Do you have a serial log of the crash??? If not, can you set up a crash
>>>>> kernel environment to capture the logs, or alternatively reproduce the
>>>>> issue on a different box which does have serial?
>>>> 
>>>> No, I don't.  I'm setup to debug ARM failures, not x86 ones.
>>> 
>>> Then alternatively can you at least give conditions that need to
>>> be met to observe the problem, for someone to repro and then
>>> debug? (The less complex the better, of course.)
>> 
>> Multiple prior messages have included statements of what I believed to be
>> the minimal case to reproduce.  Presently I believe the minimal
>> constraints are, maxmem >= host memory, memory < free Xen memory, type
>> HVM.  A minimal kr45hme.cfg file:
>> 
>> type = "hvm"
>> memory = 1024
>> maxmem = 1073741824
>> 
>> I suspect maxmem > free Xen memory may be sufficient.  The instances I
>> can be certain of have been maxmem = total host memory *7.
> 
> Can you include your Xen version and dom0 command-line?
> 
> For me, domain creation fails with an error like this:
> 
> root@immortal:/images# xl create c6-01.cfg maxmem=1073741824
> Parsing config from c6-01.cfg
> xc: error: panic: xc_dom_boot.c:120: xc_dom_boot_mem_init: can't allocate low 
> memory for domain: Out of memory
> libxl: error: libxl_dom.c:593:libxl__build_dom: xc_dom_boot_mem_init failed: 
> Cannot allocate memory
> libxl: error: libxl_create.c:1576:domcreate_rebuild_done: Domain 9:cannot 
> (re-)build domain: -3
> libxl: error: libxl_domain.c:1182:libxl__destroy_domid: Domain 9:Non-existant 
> domain
> libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain 9:Unable to 
> destroy guest
> libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain 9:Destruction of 
> domain failed
> 
> This is on staging-4.14 from a month or two ago (i.e., what I happened to 
> have on a random test  box), and `dom0_mem=1024M,max:1024M` in my 
> command-line.  That rune will give dom0 only 1GiB of RAM, but also prevent it 
> from auto-ballooning down to free up memory for the guest.

Hmm, but with that line removed, I get this:

root@immortal:/images# xl create c6-01.cfg maxmem=1073741824
Parsing config from c6-01.cfg
libxl: error: libxl_mem.c:279:libxl_set_memory_target: New target 0 for dom0 is 
below the minimum threshold
failed to free memory for the domain

Maybe the issue you’re probably facing is that “minimum threshold” safety catch 
either isn’t triggering, or is set low enough that your dom0 is OOMing trying 
to make enough memory for your VM?

That 1TiB of empty space isn’t actually free after all, even for Xen — you have 
to actually allocate p2m memory for the domain to hold all of those PoD entries.

 -George