[Xen-users] domU's being killed - strange?

[Nahuel Iglesias <niglesias@xxxxxxxxx>]

Hi all

Greetings from Patagonia Argentina - this is my first message to this list.


I'm having a weird problem: when I reboot some domU, some OTHER domU
gets killed.

Scenario:
> cat /etc/SuSE-release
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 1

> uname -sri
Linux 2.6.32.43-0.4-xen x86_64

> rpm -qa | grep xen-
kernel-xen-base-2.6.32.43-0.4.1
xen-4.0.2_21511_04-0.5.1
xen-kmp-default-4.0.2_21511_04_2.6.32.46_0.3-0.5.1
kernel-xen-2.6.32.43-0.4.1
xen-libs-4.0.2_21511_04-0.5.1
xen-doc-html-4.0.2_21511_04-0.5.1
xen-tools-4.0.2_21511_04-0.5.1
xen-doc-pdf-4.0.2_21511_04-0.5.1

I have 9 DomU's running here - most of them with several Linux
flavors, and some Windows XP SP3 too.

In most circumstances everything works OK, but sometimes, when, say,
domU "GuestC" is rebooted (e.g. when installing an OS) some other
domains get killed ("GuestA" and "GuestE" appear in grey in
virt-manager - they WHERE ON some minutes ago)

In /var/log/xen/xend.log I can see:
...(lots of DEBUG and INFO messages)
(50 seconds after last DEBUG message)
[...]
[2012-03-27 15:57:51 2225] WARNING (image:562) domain GuestA: device
model failure: pid 24438: died due to signal 6; see
/var/log/xen/qemu-dm-GuestA.log
[2012-03-27 15:57:51 2225] WARNING (XendDomainInfo:2108) Domain has
crashed: name=GuestA id=134.
[2012-03-27 15:57:51 2225] DEBUG (XendDomainInfo:3121)
XendDomainInfo.destroy: domid=134
[2012-03-27 15:57:52 2225] WARNING (image:562) domain GuestE: device
model failure: pid 6243: died due to signal 11; see
/var/log/xen/qemu-dm-GuestE.log
[2012-03-27 15:57:52 2225] WARNING (image:562) domain GuestB: device
model failure: pid 28106: malfunctioning (closed sentinel), killed;
see /var/log/xen/qemu-dm-GuestB.log
[2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:2452) Destroying device model
[2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:2459) Releasing devices
[2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:2465) Removing vif/0
[2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:1294)
XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0
[2012-03-27 15:57:52 2225] WARNING (XendDomainInfo:2108) Domain has
crashed: name=GuestB id=98.
[2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:2465) Removing vkbd/0
[2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:1294)
XendDomainInfo.destroyDevice: deviceClass = vkbd, device = vkbd/0
[2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:3121)
XendDomainInfo.destroy: domid=98
[2012-03-27 15:57:52 2225] WARNING (XendDomainInfo:2108) Domain has
crashed: name=GuestE id=109.
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:3121)
XendDomainInfo.destroy: domid=109
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2465) Removing console/0
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2452) Destroying device model
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2452) Destroying device model
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:1294)
XendDomainInfo.destroyDevice: deviceClass = console, device =
console/0
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2459) Releasing devices
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2465) Removing vif/0
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:1294)
XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2465) Removing vif/1
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:1294)
XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/1
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2465) Removing vbd/768
[2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:1294)
XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/768
[2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:2459) Releasing devices
[2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:2465) Removing vif/0
[2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:1294)
XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0
[2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:2465) Removing vif/1
[2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:1294)
XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/1
[2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:2465) Removing vkbd/0
[...]
etc.

In /var/log/xen/qemu-dm-GuestA.log:
[...]
*** glibc detected *** /usr/lib/xen/bin/qemu-dm: corrupted
double-linked list: 0x0000000000b238a0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75018)[0x7f7278b16018]
/lib64/libc.so.6(+0x755ff)[0x7f7278b165ff]
/lib64/libc.so.6(+0x77a1e)[0x7f7278b18a1e]
/lib64/libc.so.6(__libc_malloc+0x77)[0x7f7278b1b0d7]
/usr/lib/xen/bin/qemu-dm[0x482e09]
/usr/lib/xen/bin/qemu-dm[0x46c5dd]
/usr/lib/xen/bin/qemu-dm[0x46c805]
/usr/lib/xen/bin/qemu-dm[0x408772]
/usr/lib/xen/bin/qemu-dm[0x478d1a]
/usr/lib/xen/bin/qemu-dm[0x40cb0f]
/lib64/libc.so.6(__libc_start_main+0xe6)[0x7f7278abfbc6]
/usr/lib/xen/bin/qemu-dm[0x406ae9]
======= Memory map: ========
00400000-00525000 r-xp 00000000 08:03 42432
  /usr/lib/xen/bin/qemu-dm
00725000-00726000 r--p 00125000 08:03 42432
  /usr/lib/xen/bin/qemu-dm
00726000-0072c000 rw-p 00126000 08:03 42432
  /usr/lib/xen/bin/qemu-dm
[...]
Etc. (cut off in brevity's sake)

Similarly, in  /var/log/xen/qemu-dm-GuestE.log
[...]
*** glibc detected *** /usr/lib/xen/bin/qemu-dm: double free or
corruption (fasttop): 0x0000000000b126a0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75018)[0x7fd8f6dba018]
/lib64/libc.so.6(cfree+0x6c)[0x7fd8f6dbefec]
/usr/lib/xen/bin/qemu-dm[0x42f544]
/usr/lib/xen/bin/qemu-dm[0x412552]
/usr/lib/xen/bin/qemu-dm[0x4082db]
[...]
Etc.

Any ideas on what may be happening here? I'm suspicious about 64bits
kernel&libs but it shouldn't be THIS buggy anymore, isn't it?

Thanks in advance

Kind regards!

-- 
------------------------------------
Nahuel Iglesias
Bariloche - Patagonia Argentina

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users