Re: [Xen-users] Random oopsen with Xen on Ubuntu Gutsy

["Todd Deshane" <deshantm@xxxxxxxxx>]

Can you post the below to https://launchpad.net/ubuntu

I was running into something similar wasn't able to save the stack trace.

My error was when doing simple I/O, but the strace looks similar.

The gutsy kernel was working fine during alpha until they moved to 2.622+

I fought was it for awhile and got past some bugs. not all yet.

I am also using AMD64.

Todd


On Feb 5, 2008 4:24 PM, Matt Mackall <mpm@xxxxxxxxxxx> wrote:
> I've got an AMD64 machine that's exhibiting random oopses and database
> corruption using 2.6.22-14 kernel from Ubuntu Gutsy and Xen
> 3.1.0-0ubuntu18 also from Gutsy,
>
> Running the same database without Xen under heavy load appears to work
> fine. Machine has 3G of ECC and memtest86 works fine as well. Also,
> machine ran the database just fine for a year before (unfortunately
> simultaneously) switching to Gutsy and Xen.
>
> Any suggestions?
>
> MySQL said:
>
> The relevant bit of the logs is this:
>
>   Jan 10 23:46:26 vegguide mysqld[22376]: InnoDB: Database page corruption on 
> disk or a failed
>   Jan 10 23:46:26 vegguide mysqld[22376]: InnoDB: file read of page 1559.
>   Jan 10 23:46:26 vegguide mysqld[22376]: InnoDB: You may have to recover 
> from a backup.
>
> A couple of the oopsen pasted below:
>
> [297993.758358] BUG: unable to handle kernel NULL pointer dereference at 
> virtual address 00000001
> [297993.758381]  printing eip:
> [297993.758392] 006c6000 -> *pde = 00000000:6d67f001
> [297993.758398] 27f46000 -> *pme = 00000000:00000000
> [297993.758405] Oops: 0000 [#1]
> [297993.758408] SMP
> [297993.758415] Modules linked in: af_packet xt_multiport iptable_filter 
> ip_tables x_tables ipv6 evdev ext3 jbd mbcach    e dm_mirror dm_snapshot 
> dm_mod fuse apparmor commoncap
> [297993.758448] CPU:    0
> [297993.758449] EIP:    0061:[<c0176eec>]    Not tainted VLI
> [297993.758452] EFLAGS: 00010002   (2.6.22-14-xen #1)
> [297993.758468] EIP is at kmem_cache_alloc+0x5c/0xe0
> [297993.758474] eax: 00000000   ebx: 00000001   ecx: 00000000   edx: c1bfe8a0
> [297993.758480] esi: 00000000   edi: 00000000   ebp: 00000020   esp: e0003c00
> [297993.758487] ds: 007b   es: 007b   fs: 00d8  gs: 0000  ss: 0069
> [297993.758493] Process pdflush (pid: 4538, ti=e0002000 task=c210ea60 
> task.ti=e0002000)
> [297993.758498] Stack: 00000000 c1c34f60 c028b35d 00000000 c20b4b40 00000003 
> c20b4b40 c028b35d
> [297993.758515]        c12a7480 c20b4b40 00000003 c029fd40 c029f25c c013bdba 
> c03d26a0 c2048000
> [297993.758532]        c03ec4a8 c20b4b40 00000000 00000400 c0290b67 c2048000 
> 00000000 003d0900
> [297993.758549] Call Trace:
> [297993.758555]  [<c028b35d>] skb_clone+0x2d/0x250
> [297993.758567]  [<c028b35d>] skb_clone+0x2d/0x250
> [297993.758575]  [<c029fd40>] snap_rcv+0x0/0xa0
> [297993.758582]  [<c029f25c>] llc_rcv+0xdc/0x2b0
> [297993.758589]  [<c013bdba>] clocksource_get_next+0x3a/0x40
> [297993.758602]  [<c0290b67>] netif_receive_skb+0x237/0x420
> [297993.758613]  [<c027119d>] netif_poll+0x4fd/0xbf0
> [297993.758621]  [<c010891b>] sched_clock+0x3b/0x80
> [297993.758630]  [<c011e5f3>] scheduler_tick+0xf3/0x100
> [297993.758643]  [<c02930de>] net_rx_action+0xde/0x260
> [297993.758652]  [<c0127302>] __do_softirq+0x92/0x130
> [297993.758662]  [<c012742c>] do_softirq+0x8c/0x90
> [297993.758670]  [<c0106e20>] do_IRQ+0x40/0x70
> [297993.758677]  [<ee08b799>] journal_add_journal_head+0x69/0x160 [jbd]
> [297993.758699]  [<c0259946>] evtchn_do_upcall+0xb6/0xf0
> [297993.758709]  [<c01057a6>] hypervisor_callback+0x46/0x4e
> [297993.758718]  [<ee0c18cc>] walk_page_buffers+0x1c/0x70 [ext3]
> [297993.758740]  [<ee0c4ad1>] ext3_ordered_writepage+0xe1/0x1a0 [ext3]
> [297993.758759]  [<ee0c1920>] bget_one+0x0/0x10 [ext3]
> [297993.758774]  [<c0157a38>] __writepage+0x8/0x30
> [297993.758782]  [<c0157ef4>] write_cache_pages+0x214/0x310
> [297993.758791]  [<c0157a30>] __writepage+0x0/0x30
> [297993.758801]  [<c0158010>] generic_writepages+0x20/0x30
> [297993.758810]  [<c0158069>] do_writepages+0x49/0x50
> [297993.758817]  [<c01980c3>] __writeback_single_inode+0x93/0x3c0
> [297993.758828]  [<c012bb9e>] del_timer_sync+0xe/0x20
> [297993.758838]  [<c02ff246>] schedule+0x356/0x900
> [297993.758847]  [<c01f5d9d>] _atomic_dec_and_lock+0x3d/0x70
> [297993.758857]  [<c019877e>] sync_sb_inodes+0x17e/0x240
> [297993.758866]  [<c0198c49>] writeback_inodes+0x99/0xd0
> [297993.758876]  [<c0158715>] wb_kupdate+0x85/0xf0
> [297993.758885]  [<c0158ab0>] pdflush+0x0/0x260
> [297993.758892]  [<c0158bf8>] pdflush+0x148/0x260
> [297993.758900]  [<c0158690>] wb_kupdate+0x0/0xf0
> [297993.758908]  [<c0136312>] kthread+0x42/0x70
> [297993.758915]  [<c01362d0>] kthread+0x0/0x70
> [297993.758922]  [<c0105927>] kernel_thread_helper+0x7/0x10
> [297993.758930]  =======================
> [297993.758934] Code: 02 01 c6 44 02 01 01 89 f9 0f b6 f1 64 a1 08 00 42 c0 
> 8b 94 83 90 00 00 00 85 d2 74 72 8b 42 0c     85 c0 74 6b 8b 5a 0c 0f b7 42 
> 0a <8b> 04 83 89 42 0c 89 fa 84 d2 74 2e 64 a1 08 00 42 c0 c1 e0 06
> [297993.759012] EIP: [<c0176eec>] kmem_cache_alloc+0x5c/0xe0 SS:ESP 
> 0069:e0003c00
> [297993.759030] Kernel panic - not syncing: Fatal exception in interrupt
>
>
> [16785.730498] BUG: unable to handle kernel                    paging request 
> at virtual address 00100104
> [16785.730517]  printing eip:
> [16785.730528] 2d463000 -> *pde = 00000000:45af1001
> [16785.730533] 2d506000 -> *pme = 00000000:00000000
> [16785.730540] Oops: 0000 [#1]
> [16785.730543] SMP
> [16785.730550] Modules linked in: xt_multiport iptable_filter ip_tables 
> x_tables                    ipv6 evdev ext3 jbd mbcache dm_mirror dm_snapshot 
> dm_mod fuse apparmor commonca                   p
> [16785.730581] CPU:    0
> [16785.730582] EIP:    0061:[<ee0c18c9>]    Not tainted VLI
> [16785.730584] EFLAGS: 00010286   (2.6.22-14-xen #1)
> [16785.730607] EIP is at walk_page_buffers+0x19/0x70 [ext3]
> [16785.730613] eax: 00000000   ebx: fffffffe   ecx: ffffffff   edx: 00100100
> [16785.730619] esi: 00100100   edi: c18404b4   ebp: ffffffff   esp: c2149e04
> [16785.730625] ds: 007b   es: 007b   fs: 00d8  gs: 0000  ss: 0069
> [16785.730631] Process pdflush (pid: 60, ti=c2148000 task=c20fcf90 
> task.ti=c2148                   000)
> [16785.730635] Stack: 00000000 c1143210 c1830b20 c18404b4 c1143210 c1143210 
> ee0c                   4ad1 00001000
> [16785.730652]        00000000 ee0c1920 00000000 c1830b20 c2149f70 c28b46f4 
> 0000                   0000 c2149f70
> [16785.730666]        00000000 c0157a38 c1830b20 c0157ef4 00000000 0000000e 
> c015                   7a30 c28b46f4
> [16785.730681] Call Trace:
> [16785.730687]  [<ee0c4ad1>] ext3_ordered_writepage+0xe1/0x1a0 [ext3]
> [16785.730703]  [<ee0c1920>] bget_one+0x0/0x10 [ext3]
> [16785.730717]  [<c0157a38>] __writepage+0x8/0x30
> [16785.730727]  [<c0157ef4>] write_cache_pages+0x214/0x310
> [16785.730736]  [<c0157a30>] __writepage+0x0/0x30
> [16785.730745]  [<c0158010>] generic_writepages+0x20/0x30
>
> [16785.730753]  [<c0158069>] do_writepages+0x49/0x50
> [16785.730760]  [<c01980c3>] __writeback_single_inode+0x93/0x3c0
> [16785.730770]  [<c012bb9e>] del_timer_sync+0xe/0x20
> [16785.730780]  [<c02ff246>] schedule+0x356/0x900
> [16785.730789]  [<c019877e>] sync_sb_inodes+0x17e/0x240
> [16785.730799]  [<c0198c49>] writeback_inodes+0x99/0xd0
> [16785.730807]  [<c0158715>] wb_kupdate+0x85/0xf0
> [16785.730816]  [<c0158ab0>] pdflush+0x0/0x260
> [16785.730823]  [<c0158bf8>] pdflush+0x148/0x260
> [16785.730830]  [<c0158690>] wb_kupdate+0x0/0xf0
> [16785.730838]  [<c0136312>] kthread+0x42/0x70
> [16785.730846]  [<c01362d0>] kthread+0x0/0x70
> [16785.730852]  [<c0105927>] kernel_thread_helper+0x7/0x10
> [16785.730861]  =======================
> [16785.730864] Code: 74 f0 31 c0 39 cb 5b 0f 92 c0 c3 8d b4 26 00 00 00 00 55 
> 57                    89 d7 56 53 83 ec 08 89 0c 24 31 c9 89 44 24 04 8b 6a 
> 14 8d 5c 0d 00 <8b> 72 04                    3b 1c 24 76 2f 89 d8 29 e8 3b 44 
> 24 1c 73 25 8b 44 24
> [16785.730938] EIP: [<ee0c18c9>] walk_page_buffers+0x19/0x70 [ext3] SS:ESP 
> 0069:                   c2149e04
>
> --
> Mathematics is the supreme nostalgia of our time.
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users