Re: [Xen-users] domU's being killed - strange?

[Casey DeLorme <cdelorme@xxxxxxxxx>]

Greetings Nahuel,

Just letting you know your email is being received, however I do not know if I have an answer to your problem.

From the logs it looks like the bug is stemming from qemu.  My best guess would be either a resource conflict (RAM/HDD/Network/etc) or a damaged file (qemu itself or libc.so.6).

I would investigate the resources being used by the crashing machines first (what is shared).  If you compiled Xen from source I would try recompiling to rebuild qemu and reinstall libc.so.6.

On Fri, Apr 20, 2012 at 11:14 AM, Nahuel Iglesias <niglesias@xxxxxxxxx> wrote:

Hi all again

Please confirm me if my messages are making it to the list... had no
responses but I'm not sure if it is just a matter of lack o response
or not being received at all.

Thanks!

El día 27 de marzo de 2012 18:06, Nahuel Iglesias
<niglesias@xxxxxxxxx> escribió:

> Hi all
>
> Greetings from Patagonia Argentina - this is my first message to this list.
>
>
> I'm having a weird problem: when I reboot some domU, some OTHER domU
> gets killed.
>
> Scenario:
>> cat /etc/SuSE-release
> SUSE Linux Enterprise Server 11 (x86_64)
> VERSION = 11
> PATCHLEVEL = 1
>
>> uname -sri
> Linux 2.6.32.43-0.4-xen x86_64
>
>> rpm -qa | grep xen-
> kernel-xen-base-2.6.32.43-0.4.1
> xen-4.0.2_21511_04-0.5.1
> xen-kmp-default-4.0.2_21511_04_2.6.32.46_0.3-0.5.1
> kernel-xen-2.6.32.43-0.4.1
> xen-libs-4.0.2_21511_04-0.5.1
> xen-doc-html-4.0.2_21511_04-0.5.1
> xen-tools-4.0.2_21511_04-0.5.1
> xen-doc-pdf-4.0.2_21511_04-0.5.1
>
> I have 9 DomU's running here - most of them with several Linux
> flavors, and some Windows XP SP3 too.
>
> In most circumstances everything works OK, but sometimes, when, say,
> domU "GuestC" is rebooted (e.g. when installing an OS) some other
> domains get killed ("GuestA" and "GuestE" appear in grey in
> virt-manager - they WHERE ON some minutes ago)
>
> In /var/log/xen/xend.log I can see:
> ...(lots of DEBUG and INFO messages)
> (50 seconds after last DEBUG message)
> [...]
> [2012-03-27 15:57:51 2225] WARNING (image:562) domain GuestA: device
> model failure: pid 24438: died due to signal 6; see
> /var/log/xen/qemu-dm-GuestA.log
> [2012-03-27 15:57:51 2225] WARNING (XendDomainInfo:2108) Domain has
> crashed: name=GuestA id=134.
> [2012-03-27 15:57:51 2225] DEBUG (XendDomainInfo:3121)
> XendDomainInfo.destroy: domid=134
> [2012-03-27 15:57:52 2225] WARNING (image:562) domain GuestE: device
> model failure: pid 6243: died due to signal 11; see
> /var/log/xen/qemu-dm-GuestE.log
> [2012-03-27 15:57:52 2225] WARNING (image:562) domain GuestB: device
> model failure: pid 28106: malfunctioning (closed sentinel), killed;
> see /var/log/xen/qemu-dm-GuestB.log
> [2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:2452) Destroying device model
> [2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:2459) Releasing devices
> [2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:2465) Removing vif/0
> [2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:1294)
> XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0
> [2012-03-27 15:57:52 2225] WARNING (XendDomainInfo:2108) Domain has
> crashed: name=GuestB id=98.
> [2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:2465) Removing vkbd/0
> [2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:1294)
> XendDomainInfo.destroyDevice: deviceClass = vkbd, device = vkbd/0
> [2012-03-27 15:57:52 2225] DEBUG (XendDomainInfo:3121)
> XendDomainInfo.destroy: domid=98
> [2012-03-27 15:57:52 2225] WARNING (XendDomainInfo:2108) Domain has
> crashed: name=GuestE id=109.
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:3121)
> XendDomainInfo.destroy: domid=109
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2465) Removing console/0
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2452) Destroying device model
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2452) Destroying device model
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:1294)
> XendDomainInfo.destroyDevice: deviceClass = console, device =
> console/0
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2459) Releasing devices
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2465) Removing vif/0
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:1294)
> XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2465) Removing vif/1
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:1294)
> XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/1
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:2465) Removing vbd/768
> [2012-03-27 15:57:53 2225] DEBUG (XendDomainInfo:1294)
> XendDomainInfo.destroyDevice: deviceClass = vbd, device = vbd/768
> [2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:2459) Releasing devices
> [2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:2465) Removing vif/0
> [2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:1294)
> XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/0
> [2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:2465) Removing vif/1
> [2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:1294)
> XendDomainInfo.destroyDevice: deviceClass = vif, device = vif/1
> [2012-03-27 15:57:54 2225] DEBUG (XendDomainInfo:2465) Removing vkbd/0
> [...]
> etc.
>
> In /var/log/xen/qemu-dm-GuestA.log:
> [...]
> *** glibc detected *** /usr/lib/xen/bin/qemu-dm: corrupted
> double-linked list: 0x0000000000b238a0 ***
> ======= Backtrace: =========
> /lib64/libc.so.6(+0x75018)[0x7f7278b16018]
> /lib64/libc.so.6(+0x755ff)[0x7f7278b165ff]
> /lib64/libc.so.6(+0x77a1e)[0x7f7278b18a1e]
> /lib64/libc.so.6(__libc_malloc+0x77)[0x7f7278b1b0d7]
> /usr/lib/xen/bin/qemu-dm[0x482e09]
> /usr/lib/xen/bin/qemu-dm[0x46c5dd]
> /usr/lib/xen/bin/qemu-dm[0x46c805]
> /usr/lib/xen/bin/qemu-dm[0x408772]
> /usr/lib/xen/bin/qemu-dm[0x478d1a]
> /usr/lib/xen/bin/qemu-dm[0x40cb0f]
> /lib64/libc.so.6(__libc_start_main+0xe6)[0x7f7278abfbc6]
> /usr/lib/xen/bin/qemu-dm[0x406ae9]
> ======= Memory map: ========
> 00400000-00525000 r-xp 00000000 08:03 42432
>  /usr/lib/xen/bin/qemu-dm
> 00725000-00726000 r--p 00125000 08:03 42432
>  /usr/lib/xen/bin/qemu-dm
> 00726000-0072c000 rw-p 00126000 08:03 42432
>  /usr/lib/xen/bin/qemu-dm
> [...]
> Etc. (cut off in brevity's sake)
>
> Similarly, in  /var/log/xen/qemu-dm-GuestE.log
> [...]
> *** glibc detected *** /usr/lib/xen/bin/qemu-dm: double free or
> corruption (fasttop): 0x0000000000b126a0 ***
> ======= Backtrace: =========
> /lib64/libc.so.6(+0x75018)[0x7fd8f6dba018]
> /lib64/libc.so.6(cfree+0x6c)[0x7fd8f6dbefec]
> /usr/lib/xen/bin/qemu-dm[0x42f544]
> /usr/lib/xen/bin/qemu-dm[0x412552]
> /usr/lib/xen/bin/qemu-dm[0x4082db]
> [...]
> Etc.
>
> Any ideas on what may be happening here? I'm suspicious about 64bits
> kernel&libs but it shouldn't be THIS buggy anymore, isn't it?
>
> Thanks in advance
>
> Kind regards!
>
> --
> ------------------------------------
> Nahuel Iglesias
> Bariloche - Patagonia Argentina



--
------------------------------------
Nahuel Iglesias
Bariloche - Patagonia Argentina

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users