Re: [Xen-devel] can't create a vNUMA enabled PV guest

[Elena Ufimtseva <ufimtseva@xxxxxxxxx>]

On Wed, Jul 1, 2015 at 10:42 AM, Dario Faggioli
<dario.faggioli@xxxxxxxxxx> wrote:
> Hey,
>
> I know Wei is away, so I'll try to find the time to look at this myself,
> but I figured I'll let know about it, in case someone has obvious (or
> not :-D) ideas.
>
> I think I'm facing a bug that prevents creating PV guests with a vNUMA
> topology. I'm pretty sure I tested this before, while reviewing Wei's
> patches, so it must be something introduced between then and now (yes,
> we need a vNUMA OSSTest test case... I'll see about putting one
> together).
>
> So, here we are. With this as base config:
>
>   name        = 'test'
>   # Kernel, params and imags
>   kernel      = '/root/3.19.0+/vmlinuz-3.19.0+'
>   ramdisk     = '/root/3.19.0+/initrd.img-3.19.0+'
>   # CPUs and Memory and related
>   vcpus       = '4'
>   memory      = '1024'
>   vnuma = [ [ "pnode=0","size=512","vcpus=0-1","vdistances=10,20"  ],
>             [ "pnode=1","size=512","vcpus=2-3","vdistances=20,10"  ] ]
>   # Disks
>   root        = '/dev/xvda1 ro'
>   disk        = [
>                     'phy:/dev/vms/test-pv-disk,xvda1,w',
>                 ]
>   # Networking
>   dhcp        = 'dhcp'
>   vif         = [ 'mac=00:16:3E:FA:A7:9B,bridge=xenbr0' ]
>
>
>
> If I build a HVM guest, everything works:
>
> (XEN) Memory location of each domain:
> (XEN) Domain 0 (total: 129874):
> (XEN)     Node 0: 113466
> (XEN)     Node 1: 16408
> (XEN) Domain 14 (total: 262251):
> (XEN)     Node 0: 131029
> (XEN)     Node 1: 131222
> (XEN)      2 vnodes, 4 vcpus, guest physical layout:
> (XEN)          0: pnode   0, vcpus 0-1
> (XEN)            0000000000000000 - 000000001f800000
> (XEN)          1: pnode   1, vcpus 2-3
> (XEN)            000000001f800000 - 000000003f800000
>
> root@test:~# numactl --hardware
> available: 2 nodes (0-1)
> node 0 cpus: 0 1
> node 0 size: 411 MB
> node 0 free: 311 MB
> node 1 cpus: 2 3
> node 1 size: 442 MB
> node 1 free: 406 MB
> node distances:
> node   0   1
>   0:  10  20
>   1:  20  10
>
>
> If I build a PV guest, it breaks:
>
> root@Zhaman:~# xl create -c /etc/xen/test.cfg
> Parsing config from /etc/xen/test.cfg
> xc: error: panic: xc_dom_x86.c:940: arch_setup_meminit: failed to allocate 
> 0x20000 pages (v=1, p=1)
> : Internal error
> xc: error: panic: xc_dom_boot.c:155: xc_dom_boot_mem_init: can't allocate low 
> memory for domain: Out of memory
> libxl: error: libxl_dom.c:731:libxl__build_pv: xc_dom_boot_mem_init failed: 
> Device or resource busy
> libxl: error: libxl_create.c:1174:domcreate_rebuild_done: cannot (re-)build 
> domain: -3
> libxl: error: libxl.c:1586:libxl__destroy_domid: non-existant domain 15
> libxl: error: libxl.c:1544:domain_destroy_callback: unable to destroy guest 
> with domid 15
> libxl: error: libxl.c:1471:domain_destroy_cb: destruction of domain 15 failed
>
> (XEN) d0v1 Over-allocation for domain 15: 262656 > 262400
> (XEN) memory.c:155:d0v1 Could not allocate order=9 extent: id=15 memflags=210 
> (0 of 512)
> (XEN) d0v1 Over-allocation for domain 15: 262401 > 262400
> (XEN) memory.c:155:d0v1 Could not allocate order=0 extent: id=15 memflags=210 
> (256 of 131072)
>
>
> As said, I'll be looking into it in the next days. If, in the meanwhile,
> someone has any ideas, that would be much appreciated. :-)

Hi Dario!

The kernel you are running maybe missing vNUMA patch.
Konrad asked me if the patch was upstream. well, It is not, I think I
abandoned it :).
I will address latest comments and other changes in the v6 review and post it.

Elena


>
> Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel
>



-- 
Elena

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel