[Xen-devel] kernel messages: alignment check: 0000 [#1] SMP

[Teck Choon Giam <giamteckchoon@xxxxxxxxx>]

Hi,

This is just to report about the "alignment check: 0000 [#1] SMP"
kernel messages in one of my build system (domU) running Scientific
Linux 6.1.

I am compiling/rpm package for sbcl (http://sbcl.sourceforge.net/) in
my build system (domU) and hit the following kernel messages although
the domU in question still running and compilation still in progress.
I have done some searching about this and it is all related to domU
kernels 2.6.32 such as:

http://kerneltrap.org/mailarchive/reiserfs-devel/2010/12/18/6881520
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=633535
http://xen.1045712.n5.nabble.com/Alignment-check-on-domU-2-6-32-td2549903.html
http://xen.1045712.n5.nabble.com/Kernel-panic-with-2-6-32-30-under-network-activity-td3423149.html

Kernel messages:

derefnull[15862]: segfault at 0 ip 000000000804c2eb sp
00000000ffb4dfac error 4 in derefnull[8048000+57000]
divbyzero[15899] trap divide error ip:804d0be sp:ff849e9c error:0 in
divbyzero[8048000+56000]
derefnull[7328]: segfault at 0 ip 0000000000407701 sp 00007fff943b1c60
error 4 in derefnull[400000+77000]
divbyzero[7345] trap divide error ip:408d05 sp:7fff35784750 error:0 in
divbyzero[400000+76000]
derefnull[32738]: segfault at 0 ip 000000000804c2eb sp
00000000ffbdbebc error 4 in derefnull[8048000+57000]
divbyzero[32755] trap divide error ip:804d0be sp:ffc0f53c error:0 in
divbyzero[8048000+56000]
derefnull[18732]: segfault at 0 ip 0000000000407701 sp
00007fffec69eb30 error 4 in derefnull[400000+77000]
divbyzero[18841] trap divide error ip:408d05 sp:7fffed159ae0 error:0
in divbyzero[400000+76000]
derefnull[10813]: segfault at 0 ip 000000000804c2eb sp
00000000ffc17dec error 4 in derefnull[8048000+57000]
divbyzero[10827] trap divide error ip:804d0be sp:ffdb347c error:0 in
divbyzero[8048000+56000]
derefnull[27073]: segfault at 0 ip 0000000000407701 sp
00007fff37c6ba00 error 4 in derefnull[400000+77000]
divbyzero[27087] trap divide error ip:408d05 sp:7fffc93d8ab0 error:0
in divbyzero[400000+76000]
derefnull[11819]: segfault at 0 ip 000000000804c2eb sp
00000000ffbb9edc error 4 in derefnull[8048000+57000]
divbyzero[11833] trap divide error ip:804d0be sp:ff9c5a1c error:0 in
divbyzero[8048000+56000]
derefnull[28080]: segfault at 0 ip 0000000000407701 sp
00007fff5d3045d0 error 4 in derefnull[400000+77000]
divbyzero[28094] trap divide error ip:408d05 sp:7fff6369ca90 error:0
in divbyzero[400000+76000]
derefnull[22199]: segfault at 0 ip 000000000804c2eb sp
00000000ffa5f64c error 4 in derefnull[8048000+57000]
divbyzero[22213] trap divide error ip:804d0be sp:ffb2c92c error:0 in
divbyzero[8048000+56000]
derefnull[11755]: segfault at 0 ip 0000000000407701 sp
00007fffe0a4a070 error 4 in derefnull[400000+77000]
divbyzero[11768] trap divide error ip:408d05 sp:7fffaac47410 error:0
in divbyzero[400000+76000]
derefnull[3492]: segfault at 0 ip 000000000804c2eb sp 00000000ff9b0a9c
error 4 in derefnull[8048000+57000]
divbyzero[3506] trap divide error ip:804d0be sp:ffb387ac error:0 in
divbyzero[8048000+56000]
derefnull[25591]: segfault at 0 ip 0000000000407701 sp
00007fffc5e82060 error 4 in derefnull[400000+77000]
divbyzero[25607] trap divide error ip:408d05 sp:7ffff3882660 error:0
in divbyzero[400000+76000]
alignment check: 0000 [#1] SMP
last sysfs file: /sys/devices/system/cpu/cpu7/cache/index2/shared_cpu_map
CPU 7
Modules linked in: ipv6 dm_mirror dm_region_hash dm_log microcode
rtc_core rtc_lib pcspkr ext3 jbd dm_mod [last unloaded:
scsi_wait_scan]
Pid: 4002, comm: sbcl Not tainted 2.6.32.47-0.choon.sl6 #1
RIP: e030:[<ffffffff810179ff>]  [<ffffffff810179ff>]
convert_from_fxsr+0x11f/0x13d
RSP: e02b:ffff8800a8301cb8  EFLAGS: 00050246
RAX: ffff8800a8301d44 RBX: ffff8800a8301cf4 RCX: 0000000000000010
RDX: 0000000000000000 RSI: ffff8800a85c0300 RDI: ffff8800a8301cd8
RBP: ffff8800a8301cc8 R08: 000000000000c000 R09: 00000000ffffffff
R10: 0000000000000000 R11: ffff8801fdd31620 R12: ffff8801fdd31600
R13: ffff8800a8301cd8 R14: ffff8800a8301e48 R15: 0000000000000000
FS:  00007f4726298700(000f) GS:ffff880028113000(0063) knlGS:0000000000000000
CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
CR2: 00000000f7f15160 CR3: 00000000034d8000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process sbcl (pid: 4002, threadinfo ffff8800a8300000, task ffff8800a85c0300)
Stack:
 00000000f78b0224 ffff8801fdd31600 ffff8800a8301d68 ffffffff81017a4c
<0> ffff4020ffff0372 0af373e9ffffffff 0af951cc0000e033 000000000000002b
<0> ffff8800a8301f58 000000008005023b 0000000000000001 ffff88002811e0a0
Call Trace:
 [<ffffffff81017a4c>] save_i387_fxsave+0x2f/0x97
 [<ffffffff8100e57f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff8100b6cf>] ? xen_write_cr0+0x81/0x88
 [<ffffffff81017bf2>] save_i387_xstate_ia32+0x13e/0x147
 [<ffffffff81035e3b>] get_sigframe+0x8c/0xc4
 [<ffffffff81035eb0>] ia32_setup_rt_frame+0x3d/0x240
 [<ffffffff810100a7>] do_signal+0x16c/0x6b1
 [<ffffffff81034e8b>] ? sys32_clone+0x21/0x23
 [<ffffffff81034cf5>] ? ia32_ptregs_common+0x25/0x4c
 [<ffffffff81010613>] do_notify_resume+0x27/0x5f
 [<ffffffff81043e36>] ? schedule_tail+0x25/0x9c
 [<ffffffff81010d4e>] int_signal+0x12/0x17
Code: 89 47 18 8c c8 eb 1a 0f b7 86 aa 04 00 00 48 8b 96 90 04 00 00
0d 00 00 ff ff 89 47 18 48 8b 42 e0 89 47 10 48 8d 43 50 49 8b 13 <48>
89 13 66 41 8b 53 08 49 83 c3 10 66 89 53 08 48 83 c3 0a 48
RIP  [<ffffffff810179ff>] convert_from_fxsr+0x11f/0x13d
 RSP <ffff8800a8301cb8>
---[ end trace e838aa5ff4269c0b ]---

Another one happened while I am composing this email report:

alignment check: 0000 [#2] SMP
last sysfs file: /sys/devices/system/cpu/cpu7/cache/index2/shared_cpu_map
CPU 3
Modules linked in: ipv6 dm_mirror dm_region_hash dm_log microcode
rtc_core rtc_lib pcspkr ext3 jbd dm_mod [last unloaded:
scsi_wait_scan]
Pid: 23364, comm: sbcl Tainted: G      D    2.6.32.47-0.choon.sl6 #1
RIP: e030:[<ffffffff810179ff>]  [<ffffffff810179ff>]
convert_from_fxsr+0x11f/0x13d
RSP: e02b:ffff880003741cb8  EFLAGS: 00050246
RAX: ffff880003741d44 RBX: ffff880003741cf4 RCX: 0000000000000010
RDX: 0000000000000000 RSI: ffff8800a81dc600 RDI: ffff880003741cd8
RBP: ffff880003741cc8 R08: 000000000000c000 R09: 00000000ffffffff
R10: 0000000000000000 R11: ffff8801a6b77820 R12: ffff8801a6b77800
R13: ffff880003741cd8 R14: ffff880003741e48 R15: 0000000000000000
FS:  00007f944b6997a0(000f) GS:ffff8800280a3000(0063) knlGS:0000000000000000
CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
CR2: 00000000f7f1a380 CR3: 00000001d0d1f000 CR4: 0000000000002660
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process sbcl (pid: 23364, threadinfo ffff880003740000, task ffff8800a81dc600)
Stack:
 00000000f78b4224 ffff8801a6b77800 ffff880003741d68 ffffffff81017a4c
<0> ffff4020ffff0372 0af373e9ffffffff 0af951cc0000e033 000000000000002b
<0> ffff880003741f58 000000008005023b 0000000000000001 ffff8800280ae0a0
Call Trace:
 [<ffffffff81017a4c>] save_i387_fxsave+0x2f/0x97
 [<ffffffff8100e57f>] ? xen_restore_fl_direct_end+0x0/0x1
 [<ffffffff8100b6cf>] ? xen_write_cr0+0x81/0x88
 [<ffffffff81017bf2>] save_i387_xstate_ia32+0x13e/0x147
 [<ffffffff81035e3b>] get_sigframe+0x8c/0xc4
 [<ffffffff81035eb0>] ia32_setup_rt_frame+0x3d/0x240
 [<ffffffff810100a7>] do_signal+0x16c/0x6b1
 [<ffffffff81034e8b>] ? sys32_clone+0x21/0x23
 [<ffffffff81034cf5>] ? ia32_ptregs_common+0x25/0x4c
 [<ffffffff81010613>] do_notify_resume+0x27/0x5f
 [<ffffffff81043e36>] ? schedule_tail+0x25/0x9c
 [<ffffffff81010d4e>] int_signal+0x12/0x17
Code: 89 47 18 8c c8 eb 1a 0f b7 86 aa 04 00 00 48 8b 96 90 04 00 00
0d 00 00 ff ff 89 47 18 48 8b 42 e0 89 47 10 48 8d 43 50 49 8b 13 <48>
89 13 66 41 8b 53 08 49 83 c3 10 66 89 53 08 48 83 c3 0a 48
RIP  [<ffffffff810179ff>] convert_from_fxsr+0x11f/0x13d
 RSP <ffff880003741cb8>
---[ end trace e838aa5ff4269c0c ]---

Information about the running kernel in domU:

kernel: compiled from kernel.org source 2.6.32.47 with
self-patched/update ibg driver


dom0 information:

System OS: Scientific Linux 6.1

Partial related grub.conf entry:
title Scientific Linux (2.6.32.47-0.xen.pvops.choon.sl6)
        root (hd0,0)
        kernel /xen.gz dom0_mem=1024M loglvl=all guest_loglvl=all cpuidle=0
cpufreq=none
        module /vmlinuz-2.6.32.47-0.xen.pvops.choon.sl6 ro
root=UUID=1742ce45-0f9e-45da-b396-18b7e83d910e
rd_MD_UUID=8ad6e5a7:efdec690:cc2723ac:8a807302 rd_NO_LUKS rd_NO_LVM
rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc
KEYTABLE=us crashkernel=auto panic=5 panic_timeout=10
        module /initramfs-2.6.32.47-0.xen.pvops.choon.sl6.img


kernel: compiled from jeremy xen/next-2.6.32 latest commit as it is
merged with 2.6.32.47 kernel.org longterm stable branch

git url = git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
git branch = xen/next-2.6.32
git commit = faa0ece5eb2f2bc4d9b149bc46b677b1df05d332

With the following self patched or ported into the kernel:

iscsi-target 1.4.20.2
aufs 2.1-32
igb 3.2.10

Xen information in dom0:

hg_root = http://xenbits.xensource.com/staging/xen-4.1-testing.hg
hg_changeset = 23180


# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
stepping        : 7
cpu MHz         : 3192.778
cache size      : 8192 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush
acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good
nonstop_tsc aperfmperf pni pclmulqdq est ssse3 cx16 sse4_1 sse4_2
x2apic popcnt hypervisor lahf_lm ida arat
bogomips        : 6385.55
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
stepping        : 7
cpu MHz         : 3192.778
cache size      : 8192 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush
acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good
nonstop_tsc aperfmperf pni pclmulqdq est ssse3 cx16 sse4_1 sse4_2
x2apic popcnt hypervisor lahf_lm ida arat
bogomips        : 6385.55
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 2
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
stepping        : 7
cpu MHz         : 3192.778
cache size      : 8192 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush
acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good
nonstop_tsc aperfmperf pni pclmulqdq est ssse3 cx16 sse4_1 sse4_2
x2apic popcnt hypervisor lahf_lm ida arat
bogomips        : 6385.55
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
stepping        : 7
cpu MHz         : 3192.778
cache size      : 8192 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush
acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good
nonstop_tsc aperfmperf pni pclmulqdq est ssse3 cx16 sse4_1 sse4_2
x2apic popcnt hypervisor lahf_lm ida arat
bogomips        : 6385.55
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 4
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
stepping        : 7
cpu MHz         : 3192.778
cache size      : 8192 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush
acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good
nonstop_tsc aperfmperf pni pclmulqdq est ssse3 cx16 sse4_1 sse4_2
x2apic popcnt hypervisor lahf_lm ida arat
bogomips        : 6385.55
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 5
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
stepping        : 7
cpu MHz         : 3192.778
cache size      : 8192 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush
acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good
nonstop_tsc aperfmperf pni pclmulqdq est ssse3 cx16 sse4_1 sse4_2
x2apic popcnt hypervisor lahf_lm ida arat
bogomips        : 6385.55
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 6
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
stepping        : 7
cpu MHz         : 3192.778
cache size      : 8192 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush
acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good
nonstop_tsc aperfmperf pni pclmulqdq est ssse3 cx16 sse4_1 sse4_2
x2apic popcnt hypervisor lahf_lm ida arat
bogomips        : 6385.55
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 7
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
stepping        : 7
cpu MHz         : 3192.778
cache size      : 8192 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush
acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good
nonstop_tsc aperfmperf pni pclmulqdq est ssse3 cx16 sse4_1 sse4_2
x2apic popcnt hypervisor lahf_lm ida arat
bogomips        : 6385.55
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

Any idea to track this down?  This is my very first time to hit this
though.  I hope I can find a way to reproduce this easily... ...

Thanks.

Kindest regards,
Giam Teck Choon

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel