[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Kernel panic with 2.6.32-30 under network activity


  • To: xen-devel@xxxxxxxxxxxxxxxxxxx, Xen Users <xen-users@xxxxxxxxxxxxxxxxxxx>
  • From: Olivier Hanesse <olivier.hanesse@xxxxxxxxx>
  • Date: Thu, 10 Mar 2011 12:25:55 +0100
  • Cc:
  • Delivery-date: Thu, 10 Mar 2011 03:26:37 -0800
  • Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=oRO3VyV5RlR3mG6Mqvsvwu6Qbsw9lkDAhnP1kN3beh4gMh/RRcZsIwaKQoKCopifqR aPHJsdu0R+havjvXc5dbjWKPoJYz/QneecZsI4AoQrBuK7T5vi28oVz/Fivv1REdHeSR Qx56+vxfKI8rTNhBdteeUuAABVM5q0mBLOGzc=
  • List-id: Xen developer discussion <xen-devel.lists.xensource.com>

Hello,

I've got several kernel panic on a domU under network activity (multiple rsync using rsh). I didn't manage to reproduce it manually, but it happened 5times during the last month.
Each time, it is the same kernel trace.

I am using Debian 5.0.8 with kernel/hypervisor : 

ii  linux-image-2.6.32-bpo.5-amd64      2.6.32-30~bpo50+1   Linux 2.6.32 for 64-bit PCs
ii  xen-hypervisor-4.0-amd64               4.0.1-2                     The Xen Hypervisor on AMD64

Here is the  trace :

[469390.126691] alignment check: 0000 [#1] SMP 
[469390.126711] last sysfs file: /sys/devices/virtual/net/lo/operstate
[469390.126718] CPU 0 
[469390.126725] Modules linked in: snd_pcsp xen_netfront snd_pcm evdev snd_timer snd soundcore snd_page_alloc ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod xen_blkfront thermal_sys
[469390.126772] Pid: 22077, comm: rsh Not tainted 2.6.32-bpo.5-amd64 #1 
[469390.126779] RIP: e030:[<ffffffff8126093d>]  [<ffffffff8126093d>] eth_header+0x61/0x9c
[469390.126795] RSP: e02b:ffff88001ec3f9b8  EFLAGS: 00050286
[469390.126802] RAX: 00000000090f0900 RBX: 0000000000000008 RCX: ffff88001ecd0cee
[469390.126811] RDX: 0000000000000800 RSI: 000000000000000e RDI: ffff88001ecd0cee
[469390.126820] RBP: ffff8800029016d0 R08: 0000000000000000 R09: 0000000000000034
[469390.126829] R10: 000000000000000e R11: ffffffff81255821 R12: ffff880002935144
[469390.126838] R13: 0000000000000034 R14: ffff88001fe80000 R15: ffff88001fe80000
[469390.126851] FS:  00007f340c2276e0(0000) GS:ffff880002f4d000(0000) knlGS:0000000000000000
[469390.126860] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[469390.126867] CR2: 00007fffb8f33a8c CR3: 000000001d875000 CR4: 0000000000002660
[469390.126877] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[469390.126886] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[469390.126895] Process rsh (pid: 22077, threadinfo ffff88001ec3e000, task ffff88001ea61530)
[469390.126904] Stack:
[469390.126908]  0000000000000000 0000000000000000 ffff88001ecd0cfc ffff88001f1a4ae8
[469390.126921] <0> ffff880002935100 ffff880002935140 0000000000000000 ffffffff81255a20
[469390.126937] <0> 0000000000000000 ffffffff8127743d 0000000000000000 ffff88001ecd0cfc
[469390.126954] Call Trace:
[469390.126963]  [<ffffffff81255a20>] ? neigh_resolve_output+0x1ff/0x284
[469390.126974]  [<ffffffff8127743d>] ? ip_finish_output2+0x1d6/0x22b
[469390.126983]  [<ffffffff8127708f>] ? ip_queue_xmit+0x311/0x386
[469390.126994]  [<ffffffff8100dc35>] ? xen_force_evtchn_callback+0x9/0xa
[469390.127003]  [<ffffffff8100e242>] ? check_events+0x12/0x20
[469390.127013]  [<ffffffff81287a47>] ? tcp_transmit_skb+0x648/0x687
[469390.127022]  [<ffffffff8100e242>] ? check_events+0x12/0x20
[469390.127031]  [<ffffffff8100e22f>] ? xen_restore_fl_direct_end+0x0/0x1
[469390.127040]  [<ffffffff81289ec9>] ? tcp_write_xmit+0x874/0x96c
[469390.127049]  [<ffffffff8128a00e>] ? __tcp_push_pending_frames+0x22/0x53
[469390.127059]  [<ffffffff8127d409>] ? tcp_close+0x176/0x3d0
[469390.127069]  [<ffffffff81299f0c>] ? inet_release+0x4e/0x54
[469390.127079]  [<ffffffff812410d1>] ? sock_release+0x19/0x66
[469390.127087]  [<ffffffff81241140>] ? sock_close+0x22/0x26
[469390.127097]  [<ffffffff810ef879>] ? __fput+0x100/0x1af
[469390.127106]  [<ffffffff810eccb6>] ? filp_close+0x5b/0x62
[469390.127116]  [<ffffffff8104f878>] ? put_files_struct+0x64/0xc1
[469390.127127]  [<ffffffff812fbb02>] ? _spin_lock_irq+0x7/0x22
[469390.127135]  [<ffffffff81051141>] ? do_exit+0x236/0x6c6
[469390.127144]  [<ffffffff8100c241>] ? __raw_callee_save_xen_pud_val+0x11/0x1e
[469390.127154]  [<ffffffff8100e22f>] ? xen_restore_fl_direct_end+0x0/0x1
[469390.127163]  [<ffffffff8100c205>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[469390.127173]  [<ffffffff81051647>] ? do_group_exit+0x76/0x9d
[469390.127183]  [<ffffffff8105dec1>] ? get_signal_to_deliver+0x318/0x343
[469390.127193]  [<ffffffff8101004f>] ? do_notify_resume+0x87/0x73f
[469390.127202]  [<ffffffff812fbf45>] ? page_fault+0x25/0x30
[469390.127211]  [<ffffffff812fc17a>] ? error_exit+0x2a/0x60
[469390.127219]  [<ffffffff8101151d>] ? retint_restore_args+0x5/0x6
[469390.127228]  [<ffffffff8100e22f>] ? xen_restore_fl_direct_end+0x0/0x1
[469390.127240]  [<ffffffff8119564d>] ? __put_user_4+0x1d/0x30
[469390.128009]  [<ffffffff81010e0e>] ? int_signal+0x12/0x17
[469390.128009] Code: 89 e8 86 e0 66 89 47 0c 48 85 ed 75 07 49 8b ae 20 02 00 00 8b 45 00 4d 85 e4 89 47 06 66 8b 45 04 66 89 47 0a 74 12 41 8b 04 24 <89> 07 66 41 8b 44 24 04 66 89 47 04 eb 18 41 f6 86 60 01 00 00 
[469390.128009] RIP  [<ffffffff8126093d>] eth_header+0x61/0x9c
[469390.128009]  RSP <ffff88001ec3f9b8>
[469390.128009] ---[ end trace dd6b1396ef9d9a96 ]---
[469390.128009] Kernel panic - not syncing: Fatal exception in interrupt
[469390.128009] Pid: 22077, comm: rsh Tainted: G      D    2.6.32-bpo.5-amd64 #1
[469390.128009] Call Trace:
[469390.128009]  [<ffffffff812f9d03>] ? panic+0x86/0x143
[469390.128009]  [<ffffffff812fbbca>] ? _spin_unlock_irqrestore+0xd/0xe
[469390.128009]  [<ffffffff8100e22f>] ? xen_restore_fl_direct_end+0x0/0x1
[469390.128009]  [<ffffffff812fbbca>] ? _spin_unlock_irqrestore+0xd/0xe
[469390.128009]  [<ffffffff8104e387>] ? release_console_sem+0x17e/0x1af
[469390.128009]  [<ffffffff812fca65>] ? oops_end+0xa7/0xb4
[469390.128009]  [<ffffffff81012416>] ? do_alignment_check+0x88/0x92
[469390.128009]  [<ffffffff81011a75>] ? alignment_check+0x25/0x30
[469390.128009]  [<ffffffff81255821>] ? neigh_resolve_output+0x0/0x284
[469390.128009]  [<ffffffff8126093d>] ? eth_header+0x61/0x9c
[469390.128009]  [<ffffffff81260900>] ? eth_header+0x24/0x9c
[469390.128009]  [<ffffffff81255a20>] ? neigh_resolve_output+0x1ff/0x284
[469390.128009]  [<ffffffff8127743d>] ? ip_finish_output2+0x1d6/0x22b
[469390.128009]  [<ffffffff8127708f>] ? ip_queue_xmit+0x311/0x386
[469390.128009]  [<ffffffff8100dc35>] ? xen_force_evtchn_callback+0x9/0xa
[469390.128009]  [<ffffffff8100e242>] ? check_events+0x12/0x20
[469390.128009]  [<ffffffff81287a47>] ? tcp_transmit_skb+0x648/0x687
[469390.128009]  [<ffffffff8100e242>] ? check_events+0x12/0x20
[469390.128009]  [<ffffffff8100e22f>] ? xen_restore_fl_direct_end+0x0/0x1
[469390.128009]  [<ffffffff81289ec9>] ? tcp_write_xmit+0x874/0x96c
[469390.128009]  [<ffffffff8128a00e>] ? __tcp_push_pending_frames+0x22/0x53
[469390.128009]  [<ffffffff8127d409>] ? tcp_close+0x176/0x3d0
[469390.128009]  [<ffffffff81299f0c>] ? inet_release+0x4e/0x54
[469390.128009]  [<ffffffff812410d1>] ? sock_release+0x19/0x66
[469390.128009]  [<ffffffff81241140>] ? sock_close+0x22/0x26
[469390.128009]  [<ffffffff810ef879>] ? __fput+0x100/0x1af
[469390.128009]  [<ffffffff810eccb6>] ? filp_close+0x5b/0x62
[469390.128009]  [<ffffffff8104f878>] ? put_files_struct+0x64/0xc1
[469390.128009]  [<ffffffff812fbb02>] ? _spin_lock_irq+0x7/0x22
[469390.128009]  [<ffffffff81051141>] ? do_exit+0x236/0x6c6
[469390.128009]  [<ffffffff8100c241>] ? __raw_callee_save_xen_pud_val+0x11/0x1e
[469390.128009]  [<ffffffff8100e22f>] ? xen_restore_fl_direct_end+0x0/0x1
[469390.128009]  [<ffffffff8100c205>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[469390.128009]  [<ffffffff81051647>] ? do_group_exit+0x76/0x9d
[469390.128009]  [<ffffffff8105dec1>] ? get_signal_to_deliver+0x318/0x343
[469390.128009]  [<ffffffff8101004f>] ? do_notify_resume+0x87/0x73f
[469390.128009]  [<ffffffff812fbf45>] ? page_fault+0x25/0x30
[469390.128009]  [<ffffffff812fc17a>] ? error_exit+0x2a/0x60
[469390.128009]  [<ffffffff8101151d>] ? retint_restore_args+0x5/0x6
[469390.128009]  [<ffffffff8100e22f>] ? xen_restore_fl_direct_end+0x0/0x1
[469390.128009]  [<ffffffff8119564d>] ? __put_user_4+0x1d/0x30
[469390.128009]  [<ffffffff81010e0e>] ? int_signal+0x12/0x17

I found another post, which may be the same bug (same kernel, network activity ... ) :

http://jira.mongodb.org/browse/SERVER-2383

Any ideas ?

Regards

Olivier
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.