|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-devel] Linux 3.18 and Linux 4.1 reproducible OOM crashes
osstest service owner writes ("[linux-3.18 test] 97278: regressions - FAIL"):
> flight 97278 linux-3.18 real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/97278/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking,
> including tests which could not be run:
...
> test-amd64-amd64-xl 6 xen-boot fail REGR. vs. 96188
Loads of these. It seems it can't boot at all. Seems to affect amd64
dom0 kernels but not i386 or armhf. (See flight 97279 for 4.1
results.)
It seems that the oom killer is tripping. Obviously something is
seriously wrong.
It seems likely that this is a real bug, but whether it's in Xen or
Linux is not clear right now. If it's in Linux it's because a similar
patch has been backported to both.
Below is serial log extract (from the job mentioned above).
Ian.
Jul 14 00:27:33.715871 [ 22.813518] rc.local invoked oom-killer:
gfp_mask=0x84d0, order=0, oom_score_adj=0
Jul 14 00:27:33.899843 [ 22.813541] rc.local cpuset=/ mems_allowed=0
Jul 14 00:27:33.907821 [ 22.813550] CPU: 0 PID: 2676 Comm: rc.local Not
tainted 3.18.37 #1
Jul 14 00:27:33.907861 [ 22.813556] Hardware name: Intel Corporation
SandyBridge Platform/To be filled by O.E.M., BIOS
S1200BT.86B.02.00.0042.050820141549 05/08/2014
Jul 14 00:27:33.923914 [ 22.813565] 0000000000000000 ffff8800024b7968
ffffffff817dcab5 00000000000084d0
Jul 14 00:27:33.931995 [ 22.813573] 0000000000000000 ffff8800024b79c8
ffffffff8118082f ffff8800024b7988
Jul 14 00:27:33.940021 [ 22.813582] ffffffff8114c315 ffffffff817e5ee9
0000000000000001 ffff8800024b79c8
Jul 14 00:27:33.948034 [ 22.813590] Call Trace:
Jul 14 00:27:33.948065 [ 22.813600] [<ffffffff817dcab5>] dump_stack+0x7c/0x98
Jul 14 00:27:33.955838 [ 22.813609] [<ffffffff8118082f>]
dump_header.isra.11+0x8f/0x1e0
Jul 14 00:27:33.963831 [ 22.813616] [<ffffffff8114c315>] ?
__delayacct_freepages_end+0x45/0x50
Jul 14 00:27:33.963872 [ 22.813677] [<ffffffff817e5ee9>] ?
_raw_spin_unlock_irqrestore+0x29/0x90
Jul 14 00:27:33.971868 [ 22.813685] [<ffffffff8131f93e>] ?
___ratelimit+0xae/0x160
Jul 14 00:27:33.979838 [ 22.813691] [<ffffffff81180ddc>]
oom_kill_process+0x20c/0x370
Jul 14 00:27:33.987847 [ 22.813699] [<ffffffff810bbc69>] ?
has_capability_noaudit+0x19/0x20
Jul 14 00:27:33.987888 [ 22.813706] [<ffffffff811812d1>]
out_of_memory+0x211/0x330
Jul 14 00:27:33.995847 [ 22.813714] [<ffffffff81186331>]
__alloc_pages_nodemask+0xb11/0xb50
Jul 14 00:27:34.003844 [ 22.813721] [<ffffffff81186382>]
__get_free_pages+0x12/0x70
Jul 14 00:27:34.011845 [ 22.813728] [<ffffffff811863f1>]
get_zeroed_page+0x11/0x20
Jul 14 00:27:34.011882 [ 22.813735] [<ffffffff811a9c31>]
__pud_alloc+0x21/0x120
Jul 14 00:27:34.019841 [ 22.813742] [<ffffffff811ab312>]
handle_mm_fault+0x352/0xd50
Jul 14 00:27:34.027863 [ 22.813749] [<ffffffff811a550f>] ?
follow_page_mask+0x2f/0x4f0
Jul 14 00:27:34.035848 [ 22.813756] [<ffffffff811af392>] ? find_vma+0x62/0x70
Jul 14 00:27:34.035885 [ 22.813763] [<ffffffff811a5b48>]
__get_user_pages+0x178/0x660
Jul 14 00:27:34.043839 [ 22.813770] [<ffffffff811a607d>]
get_user_pages+0x4d/0x50
Jul 14 00:27:34.051847 [ 22.813776] [<ffffffff811db546>]
copy_strings.isra.26+0x176/0x310
Jul 14 00:27:34.051886 [ 22.813782] [<ffffffff811db71a>]
copy_strings_kernel+0x3a/0x60
Jul 14 00:27:34.059844 [ 22.813789] [<ffffffff811dcad8>]
do_execve_common.isra.33+0x418/0x680
Jul 14 00:27:34.067846 [ 22.813795] [<ffffffff811dd014>] SyS_execve+0x24/0x30
Jul 14 00:27:34.075839 [ 22.813801] [<ffffffff817e6b19>]
stub_execve+0x69/0xa0
Jul 14 00:27:34.075876 [ 22.813807] Mem-Info:
Jul 14 00:27:34.083841 [ 22.813810] DMA per-cpu:
Jul 14 00:27:34.083872 [ 22.813813] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 00:27:34.083905 [ 22.813817] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 00:27:34.091848 [ 22.813821] CPU 2: hi: 0, btch: 1 usd: 0
Jul 14 00:27:34.099849 [ 22.813825] CPU 3: hi: 0, btch: 1 usd: 0
Jul 14 00:27:34.099885 [ 22.813829] DMA32 per-cpu:
Jul 14 00:27:34.107837 [ 22.813833] CPU 0: hi: 186, btch: 31 usd: 88
Jul 14 00:27:34.107872 [ 22.813837] CPU 1: hi: 186, btch: 31 usd: 72
Jul 14 00:27:34.115855 [ 22.813841] CPU 2: hi: 186, btch: 31 usd: 47
Jul 14 00:27:34.115892 [ 22.813845] CPU 3: hi: 186, btch: 31 usd: 171
Jul 14 00:27:34.123844 [ 22.813852] active_anon:0 inactive_anon:0
isolated_anon:0
Jul 14 00:27:34.131837 [ 22.813852] active_file:0 inactive_file:0
isolated_file:0
Jul 14 00:27:34.131873 [ 22.813852] unevictable:69 dirty:9 writeback:0
unstable:0
Jul 14 00:27:34.139847 [ 22.813852] free:1036 slab_reclaimable:1887
slab_unreclaimable:4091
Jul 14 00:27:34.147839 [ 22.813852] mapped:4099 shmem:154 pagetables:706
bounce:0
Jul 14 00:27:34.155836 [ 22.813852] free_cma:0
Jul 14 00:27:34.155867 [ 22.813875] DMA free:1680kB min:96kB low:120kB
high:144kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:4kB isolated(anon):0kB isolated(file):0kB present:15976kB
managed:15892kB mlocked:1048kB dirty:0kB writeback:0kB mapped:408kB shmem:8kB
slab_reclaimable:92kB slab_unreclaimable:328kB kernel_stack:64kB
pagetables:148kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
Jul 14 00:27:34.195860 [ 22.813896] lowmem_reserve[]: 0 396 396 396
Jul 14 00:27:34.195898 [ 22.813908] DMA32 free:2464kB min:2496kB low:3120kB
high:3744kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:272kB isolated(anon):0kB isolated(file):0kB present:508308kB
managed:408952kB mlocked:32180kB dirty:36kB writeback:0kB mapped:15988kB
shmem:608kB slab_reclaimable:7456kB slab_unreclaimable:16036kB
kernel_stack:2720kB pagetables:2676kB unstable:0kB bounce:0kB free_cma:0kB
writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 00:27:34.243847 [ 22.813929] lowmem_reserve[]: 0 0 0 0
Jul 14 00:27:34.243882 [ 22.813936] DMA: 0*4kB 0*8kB 1*16kB (R) 0*32kB 0*64kB
1*128kB (R) 0*256kB 1*512kB (R) 1*1024kB (R) 0*2048kB 0*4096kB = 1680kB
Jul 14 00:27:34.251860 [ 22.813956] DMA32: 0*4kB 0*8kB 1*16kB (R) 1*32kB (R)
0*64kB 1*128kB (R) 1*256kB (R) 0*512kB 0*1024kB 1*2048kB (R) 0*4096kB = 2480kB
Jul 14 00:27:34.267804 [ 22.813976] 15037 total pagecache pages
Jul 14 00:27:34.267825 [ 22.813980] 0 pages in swap cache
Jul 14 00:27:34.275851 [ 22.813984] Swap cache stats: add 0, delete 0, find
0/0
Jul 14 00:27:34.283845 [ 22.813988] Free swap = 1949692kB
Jul 14 00:27:34.283879 [ 22.813991] Total swap = 1949692kB
Jul 14 00:27:34.283908 [ 22.813994] 131071 pages RAM
Jul 14 00:27:34.291912 [ 22.813997] 0 pages HighMem/MovableOnly
Jul 14 00:27:34.291946 [ 22.814000] 24860 pages reserved
Jul 14 00:27:34.299797 [ 22.814003] [ pid ] uid tgid total_vm rss
nr_ptes swapents oom_score_adj name
Jul 14 00:27:34.307798 [ 22.814013] [ 1316] 0 1316 9825 819
21 0 -1000 udevd
Jul 14 00:27:34.315792 [ 22.814021] [ 2038] 0 2038 1084 25
7 0 0 ntpdate
Jul 14 00:27:34.323800 [ 22.814027] [ 2040] 0 2040 1569 172
9 0 0 lockfile-create
Jul 14 00:27:34.331829 [ 22.814034] [ 2047] 0 2047 1060 331
7 0 0 startpar
Jul 14 00:27:34.339797 [ 22.814040] [ 2061] 0 2061 9270 668
24 0 0 rpcbind
Jul 14 00:27:34.347796 [ 22.814047] [ 2076] 106 2076 9320 700
22 0 0 rpc.statd
Jul 14 00:27:34.355808 [ 22.814054] [ 2088] 0 2088 6892 58
18 0 0 rpc.idmapd
Jul 14 00:27:34.363800 [ 22.814060] [ 2131] 0 2131 1084 410
7 0 0 rc
Jul 14 00:27:34.371807 [ 22.814066] [ 2137] 0 2137 3202 648
13 0 0 startpar
Jul 14 00:27:34.379805 [ 22.814073] [ 2160] 0 2160 65721 788
33 0 0 rsyslogd
Jul 14 00:27:34.387805 [ 22.814079] [ 2175] 0 2175 1064 394
8 0 0 acpid
Jul 14 00:27:34.395808 [ 22.814085] [ 2211] 0 2211 19431 1167
41 0 0 apache2
Jul 14 00:27:34.403799 [ 22.814091] [ 2214] 33 2214 91722 1032
69 0 0 apache2
Jul 14 00:27:34.411802 [ 22.814098] [ 2216] 33 2216 91722 1031
69 0 0 apache2
Jul 14 00:27:34.419800 [ 22.814104] [ 2422] 0 2422 4756 42
13 0 0 atd
Jul 14 00:27:34.427806 [ 22.814110] [ 2423] 0 2423 6869 590
18 0 0 cron
Jul 14 00:27:34.435804 [ 22.814117] [ 2606] 0 2606 8780 883
21 0 0 oxenstored
Jul 14 00:27:34.443846 [ 22.814123] [ 2619] 105 2619 10560 551
24 0 0 dbus-daemon
Jul 14 00:27:34.451802 [ 22.814130] [ 2624] 0 2624 13796 700
28 0 -1000 sshd
Jul 14 00:27:34.459799 [ 22.814136] [ 2631] 0 2631 22682 57
19 0 0 xenconsoled
Jul 14 00:27:34.467815 [ 22.814143] [ 2643] 104 2643 13313 828
26 0 0 exim4
Jul 14 00:27:34.475801 [ 22.814149] [ 2652] 0 2652 60164 1974
78 0 0 qemu-system-i38
Jul 14 00:27:34.483868 [ 22.814156] [ 2656] 0 2656 20648 1510
42 0 0 sshd
Jul 14 00:27:34.491859 [ 22.814162] [ 2660] 107 2660 8345 1093
21 0 0 ntpd
Jul 14 00:27:34.499864 [ 22.814169] [ 2673] 0 2673 1084 411
7 0 0 rc.local
Jul 14 00:27:34.507852 [ 22.814175] [ 2676] 0 2676 1084 44
6 0 0 rc.local
Jul 14 00:27:34.515859 [ 22.814181] [ 2677] 1000 2677 20648 946
41 0 0 sshd
Jul 14 00:27:34.523849 [ 22.814188] Out of memory: Kill process 2652
(qemu-system-i38) score 3 or sacrifice child
Jul 14 00:27:34.531862 [ 22.814194] Killed process 2652 (qemu-system-i38)
total-vm:240656kB, anon-rss:2440kB, file-rss:5456kB
Jul 14 00:27:34.539871 [ 22.822755] sshd invoked oom-killer:
gfp_mask=0x2000d0, order=0, oom_score_adj=0
Jul 14 00:27:34.547800 [ 22.822760] sshd cpuset=/ mems_allowed=0
Jul 14 00:27:34.555868 [ 22.822764] CPU: 1 PID: 2677 Comm: sshd Not tainted
3.18.37 #1
Jul 14 00:27:34.563847 [ 22.822767] Hardware name: Intel Corporation
SandyBridge Platform/To be filled by O.E.M., BIOS
S1200BT.86B.02.00.0042.050820141549 05/08/2014
Jul 14 00:27:34.571852 [ 22.822771] 0000000000000000 ffff8800024fb990
ffffffff817dcab5 00000000002000d0
Jul 14 00:27:34.579860 [ 22.822775] 0000000000000000 ffff8800024fb9f0
ffffffff8118082f ffff8800024fb9b0
Jul 14 00:27:34.587803 [ 22.822779] ffffffff8114c315 ffffffff817e5ee9
0000000000000001 ffff8800024fb9f0
Jul 14 00:27:34.595862 [ 22.822783] Call Trace:
Jul 14 00:27:34.595892 [ 22.822787] [<ffffffff817dcab5>] dump_stack+0x7c/0x98
Jul 14 00:27:34.603859 [ 22.822791] [<ffffffff8118082f>]
dump_header.isra.11+0x8f/0x1e0
Jul 14 00:27:34.611843 [ 22.822794] [<ffffffff8114c315>] ?
__delayacct_freepages_end+0x45/0x50
Jul 14 00:27:34.619891 [ 22.822798] [<ffffffff817e5ee9>] ?
_raw_spin_unlock_irqrestore+0x29/0x90
Jul 14 00:27:34.619933 [ 22.822801] [<ffffffff8131f93e>] ?
___ratelimit+0xae/0x160
Jul 14 00:27:34.627853 [ 22.822804] [<ffffffff81180ddc>]
oom_kill_process+0x20c/0x370
Jul 14 00:27:34.635900 [ 22.822808] [<ffffffff810bbc69>] ?
has_capability_noaudit+0x19/0x20
Jul 14 00:27:34.643851 [ 22.822812] [<ffffffff811812d1>]
out_of_memory+0x211/0x330
Jul 14 00:27:34.643888 [ 22.822815] [<ffffffff81186331>]
__alloc_pages_nodemask+0xb11/0xb50
Jul 14 00:27:34.651845 [ 22.822820] [<ffffffff811cb40c>] new_slab+0x2bc/0x320
Jul 14 00:27:34.660009 [ 22.822823] [<ffffffff811cc770>]
__slab_alloc.constprop.69+0x440/0x570
Jul 14 00:27:34.667913 [ 22.822828] [<ffffffff81054f3e>] ?
xen_leave_lazy_mmu+0xe/0x10
Jul 14 00:27:34.667952 [ 22.822831] [<ffffffff811b6721>] ?
anon_vma_fork+0x71/0x140
Jul 14 00:27:34.675855 [ 22.822835] [<ffffffff811b651b>] ?
anon_vma_clone+0x6b/0x200
Jul 14 00:27:34.683863 [ 22.822838] [<ffffffff811b6721>] ?
anon_vma_fork+0x71/0x140
Jul 14 00:27:34.691849 [ 22.822841] [<ffffffff811cdad6>]
kmem_cache_alloc+0x116/0x150
Jul 14 00:27:34.691887 [ 22.822844] [<ffffffff811b6721>]
anon_vma_fork+0x71/0x140
Jul 14 00:27:34.699848 [ 22.822848] [<ffffffff810b2017>]
copy_process.part.45+0x1687/0x1a50
Jul 14 00:27:34.707853 [ 22.822852] [<ffffffff810b2577>] do_fork+0xb7/0x3b0
Jul 14 00:27:34.707888 [ 22.822856] [<ffffffff811f3a34>] ?
get_unused_fd_flags+0x34/0x40
Jul 14 00:27:34.715857 [ 22.822860] [<ffffffff811f3a88>] ?
__fd_install+0x48/0x60
Jul 14 00:27:34.723872 [ 22.822862] [<ffffffff810b28f1>] SyS_clone+0x11/0x20
Jul 14 00:27:34.723910 [ 22.822866] [<ffffffff817e68c9>] stub_clone+0x69/0x90
Jul 14 00:27:34.731848 [ 22.822868] [<ffffffff817e6589>] ?
system_call_fastpath+0x12/0x17
Jul 14 00:27:34.739823 [ 22.822871] Mem-Info:
Jul 14 00:27:34.739841 [ 22.822872] DMA per-cpu:
Jul 14 00:27:34.747843 [ 22.822874] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 00:27:34.747879 [ 22.822876] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 00:27:34.755836 [ 22.822878] CPU 2: hi: 0, btch: 1 usd: 0
Jul 14 00:27:34.755870 [ 22.822880] CPU 3: hi: 0, btch: 1 usd: 0
Jul 14 00:27:34.763850 [ 22.822882] DMA32 per-cpu:
Jul 14 00:27:34.763881 [ 22.822884] CPU 0: hi: 186, btch: 31 usd: 88
Jul 14 00:27:34.771838 [ 22.822886] CPU 1: hi: 186, btch: 31 usd: 157
Jul 14 00:27:34.779837 [ 22.822888] CPU 2: hi: 186, btch: 31 usd: 47
Jul 14 00:27:34.779872 [ 22.822890] CPU 3: hi: 186, btch: 31 usd: 173
Jul 14 00:27:34.787884 [ 22.822893] active_anon:0 inactive_anon:0
isolated_anon:0
Jul 14 00:27:34.787920 [ 22.822893] active_file:0 inactive_file:0
isolated_file:0
Jul 14 00:27:34.795844 [ 22.822893] unevictable:69 dirty:9 writeback:0
unstable:0
Jul 14 00:27:34.803834 [ 22.822893] free:1036 slab_reclaimable:1887
slab_unreclaimable:4091
Jul 14 00:27:34.811846 [ 22.822893] mapped:3101 shmem:154 pagetables:650
bounce:0
Jul 14 00:27:34.811884 [ 22.822893] free_cma:0
Jul 14 00:27:34.819842 [ 22.822904] DMA free:1680kB min:96kB low:120kB
high:144kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:4kB isolated(anon):0kB isolated(file):0kB present:15976kB
managed:15892kB mlocked:1048kB dirty:0kB writeback:0kB mapped:336kB shmem:8kB
slab_reclaimable:92kB slab_unreclaimable:328kB kernel_stack:64kB
pagetables:148kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
Jul 14 00:27:34.859831 [ 22.822914] lowmem_reserve[]: 0 396 396 396
Jul 14 00:27:34.859866 [ 22.822920] DMA32 free:2464kB min:2496kB low:3120kB
high:3744kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:272kB isolated(anon):0kB isolated(file):0kB present:508308kB
managed:408952kB mlocked:32180kB dirty:36kB writeback:0kB mapped:12068kB
shmem:608kB slab_reclaimable:7456kB slab_unreclaimable:16036kB
kernel_stack:2720kB pagetables:2452kB unstable:0kB bounce:0kB free_cma:0kB
writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 00:27:34.899849 [ 22.822930] lowmem_reserve[]: 0 0 0 0
Jul 14 00:27:34.907838 [ 22.822934] DMA: 0*4kB 0*8kB 1*16kB (R) 0*32kB 0*64kB
1*128kB (R) 0*256kB 1*512kB (R) 1*1024kB (R) 0*2048kB 0*4096kB = 1680kB
Jul 14 00:27:34.915876 [ 22.822943] DMA32: 0*4kB 0*8kB 1*16kB (R) 1*32kB (R)
0*64kB 1*128kB (R) 1*256kB (R) 0*512kB 0*1024kB 1*2048kB (R) 0*4096kB = 2480kB
Jul 14 00:27:34.931842 [ 22.822953] 15037 total pagecache pages
Jul 14 00:27:34.931878 [ 22.822955] 0 pages in swap cache
Jul 14 00:27:34.939850 [ 22.822957] Swap cache stats: add 0, delete 0, find
0/0
Jul 14 00:27:34.939888 [ 22.822959] Free swap = 1949692kB
Jul 14 00:27:34.947842 [ 22.822960] Total swap = 1949692kB
Jul 14 00:27:34.947874 [ 22.822962] 131071 pages RAM
Jul 14 00:27:34.947903 [ 22.822963] 0 pages HighMem/MovableOnly
Jul 14 00:27:34.955868 [ 22.822965] 24860 pages reserved
Jul 14 00:27:34.955900 [ 22.822966] [ pid ] uid tgid total_vm rss
nr_ptes swapents oom_score_adj name
Jul 14 00:27:34.963860 [ 22.822971] [ 1316] 0 1316 9825 819
21 0 -1000 udevd
Jul 14 00:27:34.971874 [ 22.822976] [ 2038] 0 2038 1084 25
7 0 0 ntpdate
Jul 14 00:27:34.979864 [ 22.822978] [ 2040] 0 2040 1569 172
9 0 0 lockfile-create
Jul 14 00:27:34.987862 [ 22.822981] [ 2047] 0 2047 1060 331
7 0 0 startpar
Jul 14 00:27:34.995869 [ 22.822984] [ 2061] 0 2061 9270 668
24 0 0 rpcbind
Jul 14 00:27:35.003863 [ 22.822987] [ 2076] 106 2076 9320 700
22 0 0 rpc.statd
Jul 14 00:27:35.019848 [ 22.822991] [ 2088] 0 2088 6892 58
18 0 0 rpc.idmapd
Jul 14 00:27:35.027853 [ 22.822994] [ 2131] 0 2131 1084 410
7 0 0 rc
Jul 14 00:27:35.035843 [ 22.822997] [ 2137] 0 2137 3202 648
13 0 0 startpar
Jul 14 00:27:35.043857 [ 22.823000] [ 2160] 0 2160 65721 788
33 0 0 rsyslogd
Jul 14 00:27:35.051846 [ 22.823003] [ 2175] 0 2175 1064 394
8 0 0 acpid
Jul 14 00:27:35.059852 [ 22.823006] [ 2211] 0 2211 19431 1167
41 0 0 apache2
Jul 14 00:27:35.067848 [ 22.823009] [ 2214] 33 2214 91722 1032
69 0 0 apache2
Jul 14 00:27:35.075852 [ 22.823012] [ 2216] 33 2216 91722 1031
69 0 0 apache2
Jul 14 00:27:35.083847 [ 22.823015] [ 2422] 0 2422 4756 42
13 0 0 atd
Jul 14 00:27:35.091857 [ 22.823018] [ 2423] 0 2423 6869 590
18 0 0 cron
Jul 14 00:27:35.099857 [ 22.823021] [ 2606] 0 2606 8780 883
21 0 0 oxenstored
Jul 14 00:27:35.107854 [ 22.823024] [ 2619] 105 2619 10560 551
24 0 0 dbus-daemon
Jul 14 00:27:35.115866 [ 22.823027] [ 2624] 0 2624 13796 700
28 0 -1000 sshd
Jul 14 00:27:35.123797 [ 22.823030] [ 2631] 0 2631 22682 57
19 0 0 xenconsoled
Jul 14 00:27:35.131849 [ 22.823034] [ 2643] 104 2643 13313 828
26 0 0 exim4
Jul 14 00:27:35.139844 [ 22.823037] [ 2656] 0 2656 20648 1510
42 0 0 sshd
Jul 14 00:27:35.147846 [ 22.823040] [ 2660] 107 2660 8345 1093
21 0 0 ntpd
Jul 14 00:27:35.155856 [ 22.823043] [ 2673] 0 2673 1084 411
7 0 0 rc.local
Jul 14 00:27:35.163847 [ 22.823046] [ 2676] 0 2676 1084 44
6 0 0 rc.local
Jul 14 00:27:35.171852 [ 22.823049] [ 2677] 1000 2677 20648 946
41 0 0 sshd
Jul 14 00:27:35.179883 [ 22.823052] Out of memory: Kill process 2656 (sshd)
score 2 or sacrifice child
Jul 14 00:27:35.187860 [ 22.823055] Killed process 2677 (sshd)
total-vm:82592kB, anon-rss:848kB, file-rss:2936kB
Jul 14 00:27:35.195838 [ 22.825444] oxenstored invoked oom-killer:
gfp_mask=0x280da, order=0, oom_score_adj=0
Jul 14 00:27:35.203861 [ 22.825449] oxenstored cpuset=/ mems_allowed=0
Jul 14 00:27:35.203897 [ 22.825453] CPU: 2 PID: 2606 Comm: oxenstored Not
tainted 3.18.37 #1
Jul 14 00:27:35.211845 [ 22.825457] Hardware name: Intel Corporation
SandyBridge Platform/To be filled by O.E.M., BIOS
S1200BT.86B.02.00.0042.050820141549 05/08/2014
Jul 14 00:27:35.227847 [ 22.825461] 0000000000000000 ffff880008dc3ad8
ffffffff817dcab5 00000000000280da
Jul 14 00:27:35.235847 [ 22.825466] 0000000000000000 ffff880008dc3b38
ffffffff8118082f ffff880008dc3af8
Jul 14 00:27:35.243891 [ 22.825469] ffffffff8114c315 ffffffff817e5ee9
0000000000000001 ffff880008dc3b38
Jul 14 00:27:35.243937 [ 22.825473] Call Trace:
Jul 14 00:27:35.251845 [ 22.825477] [<ffffffff817dcab5>] dump_stack+0x7c/0x98
Jul 14 00:27:35.251881 [ 22.825481] [<ffffffff8118082f>]
dump_header.isra.11+0x8f/0x1e0
Jul 14 00:27:35.259846 [ 22.825484] [<ffffffff8114c315>] ?
__delayacct_freepages_end+0x45/0x50
Jul 14 00:27:35.267864 [ 22.825488] [<ffffffff817e5ee9>] ?
_raw_spin_unlock_irqrestore+0x29/0x90
Jul 14 00:27:35.275843 [ 22.825492] [<ffffffff8131f93e>] ?
___ratelimit+0xae/0x160
Jul 14 00:27:35.283904 [ 22.825494] [<ffffffff81180ddc>]
oom_kill_process+0x20c/0x370
Jul 14 00:27:35.283942 [ 22.825498] [<ffffffff810bbc69>] ?
has_capability_noaudit+0x19/0x20
Jul 14 00:27:35.291845 [ 22.825501] [<ffffffff811812d1>]
out_of_memory+0x211/0x330
Jul 14 00:27:35.299881 [ 22.825505] [<ffffffff81186331>]
__alloc_pages_nodemask+0xb11/0xb50
Jul 14 00:27:35.307834 [ 22.825509] [<ffffffff811ab988>]
handle_mm_fault+0x9c8/0xd50
Jul 14 00:27:35.307871 [ 22.825513] [<ffffffff811ea170>] ?
poll_select_copy_remaining+0x130/0x130
Jul 14 00:27:35.315942 [ 22.825517] [<ffffffff810a68dd>]
__do_page_fault+0x1bd/0x590
Jul 14 00:27:35.323846 [ 22.825521] [<ffffffff[ 23.782240] getty invoked
oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
Jul 14 00:27:35.331856 [ 23.782247] getty cpuset=/ mems_allowed=0
Jul 14 00:27:35.339842 [ 23.782252] CPU: 3 PID: 2685 Comm: getty Not tainted
3.18.37 #1
Jul 14 00:27:35.339884 [ 23.782255] Hardware name: Intel Corporation
SandyBridge Platform/To be filled by O.E.M., BIOS
S1200BT.86B.02.00.0042.050820141549 05/08/2014
Jul 14 00:27:35.355849 [ 23.782259] 0000000000000000 ffff8800158db7f8
ffffffff817dcab5 00000000000280da
Jul 14 00:27:35.363925 [ 23.782263] 0000000000000000 ffff8800158db858
ffffffff8118082f ffff8800158db818
Jul 14 00:27:35.371864 [ 23.782268] ffffffff8114c315 ffffffff817e5ee9
0000000000000001 ffff8800158db858
Jul 14 00:27:35.379855 [ 23.782272] Call Trace:
Jul 14 00:27:35.379886 [ 23.782278] [<ffffffff817dcab5>] dump_stack+0x7c/0x98
Jul 14 00:27:35.387850 [ 23.782283] [<ffffffff8118082f>]
dump_header.isra.11+0x8f/0x1e0
Jul 14 00:27:35.387888 [ 23.782287] [<ffffffff8114c315>] ?
__delayacct_freepages_end+0x45/0x50
Jul 14 00:27:35.395877 [ 23.782291] [<ffffffff817e5ee9>] ?
_raw_spin_unlock_irqrestore+0x29/0x90
Jul 14 00:27:35.403858 [ 23.782295] [<ffffffff8131f93e>] ?
___ratelimit+0xae/0x160
Jul 14 00:27:35.411839 [ 23.782298] [<ffffffff81180ddc>]
oom_kill_process+0x20c/0x370
Jul 14 00:27:35.419845 [ 23.782301] [<ffffffff811812d1>]
out_of_memory+0x211/0x330
Jul 14 00:27:35.419883 [ 23.782305] [<ffffffff81186331>]
__alloc_pages_nodemask+0xb11/0xb50
Jul 14 00:27:35.427868 [ 23.782309] [<ffffffff811ab988>]
handle_mm_fault+0x9c8/0xd50
Jul 14 00:27:35.435856 [ 23.782320] [<ffffffffa009d912>] ?
search_dir+0xc2/0x120 [ext4]
Jul 14 00:27:35.443840 [ 23.782324] [<ffffffff810a68dd>]
__do_page_fault+0x1bd/0x590
Jul 14 00:27:35.443877 [ 23.782329] [<ffffffff81233736>] ?
dquot_file_open+0x16/0x50
Jul 14 00:27:35.451860 [ 23.782334] [<ffffffffa008c953>] ?
ext4_file_open+0x43/0x180 [ext4]
Jul 14 00:27:35.459849 [ 23.782339] [<ffffffffa008c910>] ?
ext4_check_all_de+0xb0/0xb0 [ext4]
Jul 14 00:27:35.467884 [ 23.782343] [<ffffffff810a6cdc>]
do_page_fault+0x2c/0x40
Jul 14 00:27:35.467920 [ 23.782346] [<ffffffff817e81b8>] page_fault+0x28/0x30
Jul 14 00:27:35.475841 [ 23.782350] [<ffffffff811a47a8>] ?
copy_page_to_iter+0x68/0x3c0
Jul 14 00:27:35.483845 [ 23.782353] [<ffffffff8117ed88>]
generic_file_read_iter+0x278/0x5d0
Jul 14 00:27:35.491842 [ 23.782357] [<ffffffff811d4ad9>]
new_sync_read+0x79/0xb0
Jul 14 00:27:35.491880 [ 23.782360] [<ffffffff811d5303>] vfs_read+0x93/0x180
Jul 14 00:27:35.499846 [ 23.782364] [<ffffffff811d5e61>] SyS_read+0x41/0xb0
Jul 14 00:27:35.507832 [ 23.782367] [<ffffffff817e6589>]
system_call_fastpath+0x12/0x17
Jul 14 00:27:35.507869 [ 23.782370] Mem-Info:
Jul 14 00:27:35.515838 [ 23.782371] DMA per-cpu:
Jul 14 00:27:35.515869 [ 23.782373] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 00:27:35.523838 [ 23.782375] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 00:27:35.523873 [ 23.782377] CPU 2: hi: 0, btch: 1 usd: 0
Jul 14 00:27:35.531843 [ 23.782379] CPU 3: hi: 0, btch: 1 usd: 0
Jul 14 00:27:35.531878 [ 23.782381] DMA32 per-cpu:
Jul 14 00:27:35.539836 [ 23.782382] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 00:27:35.539873 [ 23.782384] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 00:27:35.547841 [ 23.782386] CPU 2: hi: 186, btch: 31 usd: 0
Jul 14 00:27:35.555833 [ 23.782388] CPU 3: hi: 186, btch: 31 usd: 0
Jul 14 00:27:35.555868 [ 23.782392] active_anon:0 inactive_anon:0
isolated_anon:0
Jul 14 00:27:35.563857 [ 23.782392] active_file:0 inactive_file:0
isolated_file:0
Jul 14 00:27:35.563895 [ 23.782392] unevictable:1 dirty:0 writeback:18
unstable:0
Jul 14 00:27:35.571859 [ 23.782392] free:1060 slab_reclaimable:1888
slab_unreclaimable:4074
Jul 14 00:27:35.579860 [ 23.782392] mapped:2923 shmem:153 pagetables:588
bounce:0
Jul 14 00:27:35.587840 [ 23.782392] free_cma:0
Jul 14 00:27:35.587872 [ 23.782403] DMA free:1668kB min:96kB low:120kB
high:144kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15976kB
managed:15892kB mlocked:1044kB dirty:0kB writeback:4kB mapped:340kB shmem:8kB
slab_reclaimable:92kB slab_unreclaimable:332kB kernel_stack:48kB
pagetables:116kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
Jul 14 00:27:35.627846 [ 23.782413] lowmem_reserve[]: 0 396 396 396
Jul 14 00:27:35.627882 [ 23.782419] DMA32 free:2572kB min:2496kB low:3120kB
high:3744kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:4kB isolated(anon):0kB isolated(file):0kB present:508308kB
managed:408952kB mlocked:31912kB dirty:0kB writeback:68kB mapped:11352kB
shmem:604kB slab_reclaimable:7460kB slab_unreclaimable:15964kB
kernel_stack:2400kB pagetables:2236kB unstable:0kB bounce:0kB free_cma:0kB
writeback_tmp:0kB pages_scanned:36 all_unreclaimable? yes
Jul 14 00:27:35.675846 [ 23.782431] lowmem_reserve[]: 0 0 0 0
Jul 14 00:27:35.675882 [ 23.782433] DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB (R) 0*256kB 1*512kB (R) 1*1024kB (R) 0*2048kB 0*4096kB = 1664kB
Jul 14 00:27:35.683866 [ 23.782442] DMA32: 29*4kB (UR) 0*8kB 3*16kB (R)
0*32kB 1*64kB (R) 1*128kB (R) 1*256kB (R) 0*512kB 0*1024kB 1*2048kB (R)
0*4096kB = 2660kB
Jul 14 00:27:35.699851 [ 23.782452] 15076 total pagecache pages
Jul 14 00:27:35.699885 [ 23.782455] 0 pages in swap cache
Jul 14 00:27:35.707849 [ 23.782465] Swap cache stats: add 0, delete 0, find
0/0
Jul 14 00:27:35.715836 [ 23.782468] Free swap = 1949692kB
Jul 14 00:27:35.715869 [ 23.782472] Total swap = 1949692kB
Jul 14 00:27:35.715899 [ 23.782476] 131071 pages RAM
Jul 14 00:27:35.723847 [ 23.782480] 0 pages HighMem/MovableOnly
Jul 14 00:27:35.723882 [ 23.782484] 24860 pages reserved
Jul 14 00:27:35.731838 [ 23.782487] [ pid ] uid tgid total_vm rss
nr_ptes swapents oom_score_adj name
Jul 14 00:27:35.739850 [ 23.782495] [ 1316] 0 1316 9825 819
21 0 -1000 udevd
Jul 14 00:27:35.747851 [ 23.782501] [ 2038] 0 2038 1084 25
7 0 0 ntpdate
Jul 14 00:27:35.755844 [ 23.782506] [ 2040] 0 2040 1569 172
9 0 0 lockfile-create
Jul 14 00:27:35.763856 [ 23.782509] [ 2047] 0 2047 1060 331
7 0 0 startpar
Jul 14 00:27:35.771890 [ 23.782512] [ 2061] 0 2061 9270 668
24 0 0 rpcbind
Jul 14 00:27:35.779829 [ 23.782516] [ 2076] 106 2076 9320 700
22 0 0 rpc.statd
Jul 14 00:27:35.787856 [ 23.782519] [ 2088] 0 2088 6892 58
18 0 0 rpc.idmapd
Jul 14 00:27:35.795799 [ 23.782522] [ 2160] 0 2160 65721 785
33 0 0 rsyslogd
Jul 14 00:27:35.803857 [ 23.782525] [ 2175] 0 2175 1064 394
8 0 0 acpid
Jul 14 00:27:35.811854 [ 23.782528] [ 2211] 0 2211 19431 1167
41 0 0 apache2
Jul 14 00:27:35.820250 [ 23.782532] [ 2216] 33 2216 91722 1031
69 0 0 apache2
Jul 14 00:27:35.827884 [ 23.782534] [ 2422] 0 2422 4756 42
13 0 0 atd
Jul 14 00:27:35.835897 [ 23.782537] [ 2423] 0 2423 6869 590
18 0 0 cron
Jul 14 00:27:35.843893 [ 23.782541] [ 2606] 0 2606 8780 883
21 0 0 oxenstored
Jul 14 00:27:35.851883 [ 23.782544] [ 2619] 105 2619 10560 551
24 0 0 dbus-daemon
Jul 14 00:27:35.859858 [ 23.782547] [ 2624] 0 2624 13796 700
28 0 -1000 sshd
Jul 14 00:27:35.867866 [ 23.782550] [ 2631] 0 2631 22682 57
19 0 0 xenconsoled
Jul 14 00:27:35.875902 [ 23.782554] [ 2643] 104 2643 13313 828
26 0 0 exim4
Jul 14 00:27:35.883887 [ 23.782557] [ 2660] 107 2660 8346 1093
21 0 0 ntpd
Jul 14 00:27:35.891912 [ 23.782560] [ 2681] 0 2681 2650 399
10 0 0 getty
Jul 14 00:27:35.899869 [ 23.782563] [ 2682] 0 2682 3166 417
12 0 0 getty
Jul 14 00:27:35.907853 [ 23.782566] [ 2683] 0 2683 1063 173
7 0 0 getty
Jul 14 00:27:35.915859 [ 23.782569] [ 2684] 0 2684 3166 416
12 0 0 getty
Jul 14 00:27:35.923807 [ 23.782572] [ 2685] 0 2685 3560 425
12 0 0 getty
Jul 14 00:27:35.931919 [ 23.782575] [ 2686] 0 2686 3166 425
11 0 0 getty
Jul 14 00:27:35.939914 [ 23.782578] [ 2687] 0 2687 1591 404
8 0 0 getty
Jul 14 00:27:35.947888 [ 23.782581] [ 2688] 0 2688 3166 437
12 0 0 getty
Jul 14 00:27:35.956047 [ 23.782587] [ 2690] 0 2690 3166 35
9 0 0 getty
Jul 14 00:27:35.964079 [ 23.782592] [ 2691] 0 2691 3166 34
9 0 0 getty
Jul 14 00:27:35.972103 [ 23.782597] [ 2693] 0 2693 9825 195
19 0 -1000 udevd
Jul 14 00:27:35.980089 [ 23.782601] [ 2694] 0 2694 9825 195
18 0 -1000 udevd
Jul 14 00:27:35.987933 [ 23.782604] Out of memory: Kill process 2211
(apache2) score 1 or sacrifice child
Jul 14 00:27:35.996043 [ 23.782607] Killed process 2216 (apache2)
total-vm:366888kB, anon-rss:1556kB, file-rss:2568kB
Jul 14 00:27:36.004061 [ 23.790140] getty invoked oom-killer:
gfp_mask=0x200da, order=0, oom_score_adj=0
Jul 14 00:27:36.011954 [ 23.790145] getty cpuset=/ mems_allowed=0
Jul 14 00:27:36.019911 [ 23.790150] CPU: 0 PID: 2681 Comm: getty Not tainted
3.18.37 #1
Jul 14 00:27:36.019951 [ 23.790152] Hardware name: Intel Corporation
SandyBridge Platform/To be filled by O.E.M., BIOS
S1200BT.86B.02.00.0042.050820141549 05/08/2014
Jul 14 00:27:36.035934 [ 23.790156] 0000000000000000 ffff88001220ba78
ffffffff817dcab5 00000000000200da
Jul 14 00:27:36.043951 [ 23.790161] 0000000000000000 ffff88001220bad8
ffffffff8118082f ffff88001220ba98
Jul 14 00:27:36.051973 [ 23.790166] ffffffff8114c315 ffffffff817e5ee9
0000000000000001 ffff88001220bad8
Jul 14 00:27:36.059965 [ 23.790170] Call Trace:
Jul 14 00:27:36.059997 [ 23.790174] [<ffffffff817dcab5>] dump_stack+0x7c/0x98
Jul 14 00:27:36.067938 [ 23.790178] [<ffffffff8118082f>]
dump_header.isra.11+0x8f/0x1e0
Jul 14 00:27:36.067977 [ 23.790181] [<ffffffff8114c315>] ?
__delayacct_freepages_end+0x45/0x50
Jul 14 00:27:36.076024 [ 23.790185] [<ffffffff817e5ee9>] ?
_raw_spin_unlock_irqrestore+0x29/0x90
Jul 14 00:27:36.083947 [ 23.790188] [<ffffffff8131f93e>] ?
___ratelimit+0xae/0x160
Jul 14 00:27:36.092010 [ 23.790191] [<ffffffff81180ddc>]
oom_kill_process+0x20c/0x370
Jul 14 00:27:36.099947 [ 23.790195] [<ffffffff811812d1>]
out_of_memory+0x211/0x330
Jul 14 00:27:36.099991 [ 23.790198] [<ffffffff81186331>]
__alloc_pages_nodemask+0xb11/0xb50
Jul 14 00:27:36.107888 [ 23.790202] [<ffffffff811a9757>]
do_cow_fault.isra.96+0x47/0x2a0
Jul 14 00:27:36.116087 [ 23.790207] [<ffffffff81054bf9>] ?
__raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 00:27:36.123936 [ 23.790210] [<ffffffff811ab675>]
handle_mm_fault+0x6b5/0xd50
Jul 14 00:27:36.124004 [ 23.790214] [<ffffffff810a68dd>]
__do_page_fault+0x1bd/0x590
Jul 14 00:27:36.132061 [ 23.790217] [<ffffffff81198939>] ?
vm_mmap_pgoff+0x99/0xc0
Jul 14 00:27:36.140090 [ 23.790220] [<ffffffff810a6cdc>]
do_page_fault+0x2c/0x40
Jul 14 00:27:36.148022 [ 23.790224] [<ffffffff817e81b8>] page_fault+0x28/0x30
Jul 14 00:27:36.148057 [ 23.790226] Mem-Info:
Jul 14 00:27:36.148083 [ 23.790227] DMA per-cpu:
Jul 14 00:27:36.155974 [ 23.790229] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 00:27:36.155995 [ 23.790231] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 00:27:36.164132 [ 23.790233] CPU 2: hi: 0, btch: 1 usd: 0
Jul 14 00:27:36.172145 [ 23.790235] CPU 3: hi: 0, btch: 1 usd: 0
Jul 14 00:27:36.172208 [ 23.790237] DMA32 per-cpu:
Jul 14 00:27:36.180000 [ 23.790238] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 00:27:36.180049 [ 23.790241] CPU 1: hi: 186, btch: 31 usd: 128
Jul 14 00:27:36.187954 [ 23.790243] CPU 2: hi: 186, btch: 31 usd: 0
Jul 14 00:27:36.187996 [ 23.790245] CPU 3: hi: 186, btch: 31 usd: 0
Jul 14 00:27:36.195965 [ 23.790248] active_anon:0 inactive_anon:0
isolated_anon:0
Jul 14 00:27:36.203964 [ 23.790248] active_file:0 inactive_file:0
isolated_file:0
Jul 14 00:27:36.204010 [ 23.790248] unevictable:1 dirty:4 writeback:0
unstable:0
Jul 14 00:27:36.211969 [ 23.790248] free:1012 slab_reclaimable:1888
slab_unreclaimable:4067
Jul 14 00:27:36.219948 [ 23.790248] mapped:2877 shmem:154 pagetables:521
bounce:0
Jul 14 00:27:36.227943 [ 23.790248] free_cma:0
Jul 14 00:27:36.227982 [ 23.790259] DMA free:1680kB min:96kB low:120kB
high:144kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15976kB
managed:15892kB mlocked:1044kB dirty:0kB writeback:0kB mapped:340kB shmem:8kB
slab_reclaimable:92kB slab_unreclaimable:332kB kernel_stack:48kB
pagetables:84kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
Jul 14 00:27:36.267995 [ 23.790269] lowmem_reserve[]: 0 396 396 396
Jul 14 00:27:36.268052 [ 23.790275] DMA32 free:2368kB min:2496kB low:3120kB
high:3744kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:4kB isolated(anon):0kB isolated(file):0kB present:508308kB
managed:408952kB mlocked:319
Jul 14 00:28:18.168603 <client 0x1c9b400 connected - now 1 clients>
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |