Good Morning,
Paul Durrant tould us to send our problem to xen users instead of devs, so i open now a problem case to this mailinglist.
Cause of the not longer support of xen 4.4 hypervisor, we actually upgrade all of our xen hosts from 4.4 - debian 8 to 4.8 - debian 9.
Till this update, we run a host for example with 16 GB memory for dom0 with about 93 VMs. Till the upgrade, all memory usage was fine. The Host use about 1.4 - 3 GB memory of the allocated 16 GB.
At each host, we mix HVM and Para VMs. After the Upgrade, the HVM VMs constantly use more and more memory. About 100 MB more each 2-3 Minutes until the Host swaps. The Problem is only with HVM VMs, Para is all fine.
top - 09:40:43 up 1 day, 2:28, 1 user, load average: 0,94, 1,03, 1,27
Tasks: 1313 total, 5 running, 1308 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0,2 us, 0,8 sy, 0,0 ni, 98,3 id, 0,0 wa, 0,0 hi, 0,3 si, 0,5 st KiB Mem : 30315388 total, 10791316 free, 18483368 used, 1040704 buff/cache KiB Swap: 15998972 total, 15919092 free, 79880 used. 11492116 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
17962 root 20 0 1202580 680040 17356 R 5,2 2,2 1:20.06 qemu-system-i38
21679 root 20 0 1454316 922580 17760 R 5,2 3,0 1:26.99 qemu-system-i38
27772 root 20 0 1635108 1,082g 17524 S 5,2 3,7 1:27.50 qemu-system-i38
29731 root 20 0 1374844 896052 17016 S 5,2 3,0 1:17.82 qemu-system-i38
14209 root 20 0 1130476 597120 17724 S 4,9 2,0 1:17.24 qemu-system-i38
19846 root 20 0 1417076 921928 16952 S 4,6 3,0 1:31.96 qemu-system-i38
4830 root 20 0 2092624 1,496g 17640 S 3,9 5,2 1:53.88 qemu-system-i38
18897 root 20 0 2013120 1,353g 17932 S 3,9 4,7 1:30.63 qemu-system-i38
7832 root 20 0 46296 5160 3176 R 2,3 0,0 0:00.38 top
31832 root 20 0 1373044 835140 17688 S 2,3 2,8 0:48.82 qemu-system-i38
28744 root 20 0 1053868 530680 17944 S 2,0 1,8 0:34.11 qemu-system-i38
13307 root 20 0 913684 424984 17168 S 1,6 1,4 0:27.57 qemu-system-i38
15248 root 20 0 1411316 887892 17608 S 1,6 2,9 0:43.97 qemu-system-i38
16135 root 20 0 1204240 640644 17776 S 1,6 2,1 0:37.28 qemu-system-i38
20763 root 20 0 1036288 513848 17484 S 1,6 1,7 0:35.76 qemu-system-i38
22663 root 20 0 851712 301236 17588 S 1,6 1,0 0:28.24 qemu-system-i38
24849 root 20 0 1164908 644736 17824 S 1,6 2,1 0:39.93 qemu-system-i38
25871 root 20 0 1113684 571548 17616 S 1,6 1,9 0:37.14 qemu-system-i38
26840 root 20 0 1045604 515216 17888 S 1,6 1,7 0:35.42 qemu-system-i38
30693 root 20 0 2329944 1,734g 17644 S 1,6 6,0 1:32.11 qemu-system-i38
23743 root 20 0 1470544 929708 17500 S 1,3 3,1 0:47.23 qemu-system-i38
The config file of one HVM guest:
#kernel = "hvmloader"
builder='hvm'
memory = 512
maxmem = 512
shadow_memory = 8
name = "vmanager1157"
vif = [ 'vifname=vmanager1157, rate=100Mb/s, bridge=xenbr0.240, mac=xxxx, ip=xxxx xxxx' ] vif_other_config = [ ‚xxxx‘, 'tbf', 'rate=100Mb/s', 'bps_read=150Mb/s', 'bps_write=150Mb/s', 'iops_read=150000IOPS', 'iops_write=150000IOPS' ]
disk = [ 'phy:/dev/vm/vmanager1157-root,xvda,w', 'file:/root/vmanager/iso/CentOS-7.0-1406-x86_64-NetInstall.iso,xvdc:cdrom,r' ] boot="cd"
vcpus = 1
sdl=0
vnc=1
vnclisten="0.0.0.0"
vncdisplay=69
vncpasswd=xxxx
vncunused=0
stdvga=0
serial='pty'
usbdevice='tablet'
_on_poweroff_ = 'destroy'
on_reboot = 'restart'
on_crash = 'destroy'
Actually we use Kernel Version 4.12.10. So the newest Kernel with Xen Support.
Paul Durrant tould us that the main change from 4.4 to 4.8 is the QEMU change from traditional to upstream. The problem is, we do not build Xen ourself with the xen package, we use the Debian package from Debian 9 and there is no traditional
QEMU option inside.
root@v34:/etc/xen# xl create vmanager1866.cfg Parsing config from vmanager1866.cfg
libxl: error: libxl_dm.c:2024:libxl__spawn_local_dm: device model /usr/lib/xen-4.8/bin/qemu-dm is not executable: No such file or directory
libxl: error: libxl_dm.c:2189:device_model_spawn_outcome: (null): spawn failed (rc=-3)
libxl: error: libxl_create.c:1504:domcreate_devmodel_started: device model did not start: -3
libxl: error: libxl.c:1575:libxl__destroy_domid: non-existant domain 67
libxl: error: libxl.c:1534:domain_destroy_callback: unable to destroy guest with domid 67
libxl: error: libxl.c:1463:domain_destroy_cb: destruction of domain 67 failed
The Problem with the new QEMU is not only the Memory usage of HVM Guests, it is also that the HVM guests are very very instabil, crashing often with different OS Versions… It is crazy to sell this solution as stable and usable virtualisation.
Mit freundlichen Grüßen
Michael Schinzel
- Geschäftsführer -
|
|
IP-Projects GmbH & Co. KG
Am Vogelherd 14
D - 97295 Waldbrunn
|
Telefon: 09306 - 76499-0
FAX: 09306 - 76499-15
E-Mail: info@xxxxxxxxxxxxxx
|
Geschäftsführer: Michael Schinzel
Registergericht Würzburg: HRA 6798
Komplementär: IP-Projects Verwaltungs GmbH
|
