[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] xen 4.1.4 problem with vCPUs assignment to physical CPUs


  • To: xen-users@xxxxxxxxxxxxx
  • From: Grzegorz Bakalarski <grzesb@xxxxxxxxxx>
  • Date: Tue, 07 Jan 2014 17:21:19 +0100
  • Delivery-date: Tue, 07 Jan 2014 18:02:33 +0000
  • List-id: Xen user discussion <xen-users.lists.xen.org>

Hi

We have two identical hardware machines (Dell R815 AMD Opteron 4x 6172 + 192GB RAM each).

One is running older Debian squeezy with Xen 4.1.2 (from backports) with home made kernel based
on 3.2.0-4 linux kernel. This behaves normally

Other one is running new Debian wheezy with Xen 4.1.4 with the same home made kernel as first one.
This behaves abnormally.

Both machines run few PV guests. It really few - older/normal runs 2 VMs (16 & 2 vCPUs == total 18, and newer/abnormal runs 5 VMs ( 16 & 8 & 4 & 6 & 8 == total 42)

What is abnormal???? We test bigest (i.e. 16 CPU) guests.
When running full load on normal VM, i.e. 16 number crunching proceses
xentop shows utilization of 1600% CPU for the normal VM. Also when fully loaded xm vcpu-list shows that vCPUs are placed on different physical CPUs and Steal Time shown by top
is under 1%.

The same run on newer xen results in only 1200% CPU utilization (regardless of 16 or 32 or 128 running heavy processes), and steal time varies from 20 to 50%. xm vcpu-list shows that even for very overloaded guest 8 vCPUs are correctly assigned to 8 physical CPUs but other 8 vCPUs are by pair assigned to 4 physical CPUs (i.e. 2 vCPUs per one physical CPUs). BTW other guess machines are very very light loaded (less than 100% i.e. less then one vCPU)

Both domU see 16 cpu.

Home made kernel is almost debian kernel with NUMA switched off (in kernel).

We restarted suspicious VM but no change. We cannot restart dom0 at the moment ...

Any hints what going wrong? Will cpu pinnig help? Would creating cpu pool help ????

TIA

GB

PS


Here xm info -n for both machines:

^^^^^^^ normal ^^^^^^^^^^^^^^^^^^^^
host                   : build1-dom0
release                : 3.2.0-1-amd64
version                : #1 SMP Wed Feb 29 15:14:57 UTC 2012
machine                : x86_64
nr_cpus                : 48
nr_nodes               : 8
cores_per_socket       : 12
threads_per_core       : 1
cpu_mhz                : 2200
hw_caps : 178bf3ff:efd3fbff:00000000:00001710:00802001:00000000:000837ff:00000000
virt_caps              : hvm
total_memory           : 196598
free_memory            : 119218
free_cpus              : 0
cpu_topology           :
cpu:    core    socket     node
  0:       0        0        0
  1:       1        0        0
  2:       2        0        0
  3:       3        0        0
  4:       4        0        0
  5:       5        0        0
  6:       6        0        1
  7:       7        0        1
  8:       8        0        1
  9:       9        0        1
 10:      10        0        1
 11:      11        0        1
 12:       0        1        2
 13:       1        1        2
 14:       2        1        2
 15:       3        1        2
 16:       4        1        2
 17:       5        1        2
 18:       6        1        3
 19:       7        1        3
 20:       8        1        3
 21:       9        1        3
 22:      10        1        3
 23:      11        1        3
 24:       0        2        4
 25:       1        2        4
 26:       2        2        4
 27:       3        2        4
 28:       4        2        4
 29:       5        2        4
 30:       6        2        5
 31:       7        2        5
 32:       8        2        5
 33:       9        2        5
 34:      10        2        5
 35:      11        2        5
 36:       0        3        6
 37:       1        3        6
 38:       2        3        6
 39:       3        3        6
 40:       4        3        6
 41:       5        3        6
 42:       6        3        7
 43:       7        3        7
 44:       8        3        7
 45:       9        3        7
 46:      10        3        7
 47:      11        3        7
numa_info              : none
xen_major              : 4
xen_minor              : 1
xen_extra              : .2
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : unavailable
xen_commandline : placeholder com2=115200,8n1 vga=mode-0x0319 console=com2,vga guest_loglvl=error dom0_mem=6144M
cc_compiler            : gcc version 4.6.2 (Debian 4.6.2-6)
cc_compile_by          : waldi
cc_compile_domain      : debian.org
cc_compile_date        : Sat Dec 10 19:58:21 UTC 2011
xend_config_format     : 4




============ABnormal==============
host                   : prod3-dom0
release                : 3.2.0-1-amd64
version                : #1 SMP Wed Feb 29 15:14:57 UTC 2012
machine                : x86_64
nr_cpus                : 48
nr_nodes               : 8
cores_per_socket       : 12
threads_per_core       : 1
cpu_mhz                : 2200
hw_caps : 178bf3ff:efd3fbff:00000000:00001310:00802001:00000000:000837ff:00000000
virt_caps              : hvm
total_memory           : 196598
free_memory            : 57776
free_cpus              : 0
cpu_topology           :
cpu:    core    socket     node
  0:       0        0        0
  1:       1        0        0
  2:       2        0        0
  3:       3        0        0
  4:       4        0        0
  5:       5        0        0
  6:       6        0        1
  7:       7        0        1
  8:       8        0        1
  9:       9        0        1
 10:      10        0        1
 11:      11        0        1
 12:       0        1        2
 13:       1        1        2
 14:       2        1        2
 15:       3        1        2
 16:       4        1        2
 17:       5        1        2
 18:       6        1        3
 19:       7        1        3
 20:       8        1        3
 21:       9        1        3
 22:      10        1        3
 23:      11        1        3
 24:       0        2        4
 25:       1        2        4
 26:       2        2        4
 27:       3        2        4
 28:       4        2        4
 29:       5        2        4
 30:       6        2        5
 31:       7        2        5
 32:       8        2        5
 33:       9        2        5
 34:      10        2        5
 35:      11        2        5
 36:       0        3        6
 37:       1        3        6
 38:       2        3        6
 39:       3        3        6
 40:       4        3        6
 41:       5        3        6
 42:       6        3        7
 43:       7        3        7
 44:       8        3        7
 45:       9        3        7
 46:      10        3        7
 47:      11        3        7
numa_info              : none
xen_major              : 4
xen_minor              : 1
xen_extra              : .4
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : unavailable
xen_commandline : placeholder com2=115200,8n1 vga=mode-0x0319 console=com2,vga guest_loglvl=error dom0_mem=6144M
cc_compiler            : gcc version 4.7.2 (Debian 4.7.2-5)
cc_compile_by          : carnil
cc_compile_domain      : debian.org
cc_compile_date        : Sun May  5 14:44:49 UTC 2013
xend_config_format     : 4

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.