[Xen-devel] can't create a vNUMA enabled PV guest

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

Hey,

I know Wei is away, so I'll try to find the time to look at this myself,
but I figured I'll let know about it, in case someone has obvious (or
not :-D) ideas.

I think I'm facing a bug that prevents creating PV guests with a vNUMA
topology. I'm pretty sure I tested this before, while reviewing Wei's
patches, so it must be something introduced between then and now (yes,
we need a vNUMA OSSTest test case... I'll see about putting one
together).

So, here we are. With this as base config:

  name        = 'test'
  # Kernel, params and imags
  kernel      = '/root/3.19.0+/vmlinuz-3.19.0+'
  ramdisk     = '/root/3.19.0+/initrd.img-3.19.0+'
  # CPUs and Memory and related
  vcpus       = '4'
  memory      = '1024'
  vnuma = [ [ "pnode=0","size=512","vcpus=0-1","vdistances=10,20"  ],
            [ "pnode=1","size=512","vcpus=2-3","vdistances=20,10"  ] ]
  # Disks
  root        = '/dev/xvda1 ro'
  disk        = [
                    'phy:/dev/vms/test-pv-disk,xvda1,w',
                ]
  # Networking
  dhcp        = 'dhcp'
  vif         = [ 'mac=00:16:3E:FA:A7:9B,bridge=xenbr0' ]



If I build a HVM guest, everything works:

(XEN) Memory location of each domain:
(XEN) Domain 0 (total: 129874):
(XEN)     Node 0: 113466
(XEN)     Node 1: 16408
(XEN) Domain 14 (total: 262251):
(XEN)     Node 0: 131029
(XEN)     Node 1: 131222
(XEN)      2 vnodes, 4 vcpus, guest physical layout:
(XEN)          0: pnode   0, vcpus 0-1 
(XEN)            0000000000000000 - 000000001f800000
(XEN)          1: pnode   1, vcpus 2-3
(XEN)            000000001f800000 - 000000003f800000

root@test:~# numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1
node 0 size: 411 MB
node 0 free: 311 MB
node 1 cpus: 2 3
node 1 size: 442 MB
node 1 free: 406 MB
node distances:
node   0   1 
  0:  10  20 
  1:  20  10 


If I build a PV guest, it breaks:

root@Zhaman:~# xl create -c /etc/xen/test.cfg
Parsing config from /etc/xen/test.cfg
xc: error: panic: xc_dom_x86.c:940: arch_setup_meminit: failed to allocate 
0x20000 pages (v=1, p=1)
: Internal error
xc: error: panic: xc_dom_boot.c:155: xc_dom_boot_mem_init: can't allocate low 
memory for domain: Out of memory
libxl: error: libxl_dom.c:731:libxl__build_pv: xc_dom_boot_mem_init failed: 
Device or resource busy
libxl: error: libxl_create.c:1174:domcreate_rebuild_done: cannot (re-)build 
domain: -3
libxl: error: libxl.c:1586:libxl__destroy_domid: non-existant domain 15
libxl: error: libxl.c:1544:domain_destroy_callback: unable to destroy guest 
with domid 15
libxl: error: libxl.c:1471:domain_destroy_cb: destruction of domain 15 failed

(XEN) d0v1 Over-allocation for domain 15: 262656 > 262400
(XEN) memory.c:155:d0v1 Could not allocate order=9 extent: id=15 memflags=210 
(0 of 512)
(XEN) d0v1 Over-allocation for domain 15: 262401 > 262400
(XEN) memory.c:155:d0v1 Could not allocate order=0 extent: id=15 memflags=210 
(256 of 131072)


As said, I'll be looking into it in the next days. If, in the meanwhile,
someone has any ideas, that would be much appreciated. :-)

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel