Re: [Xen-devel] [xen-unstable test] 116178: regressions - FAIL

[Julien Grall <julien.grall@xxxxxxxxxx>]

Hi,

On 11/15/2017 11:29 AM, osstest service owner wrote:
> flight 116178 xen-unstable real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/116178/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>   test-armhf-armhf-libvirt-raw 15 guest-start/debian.repeat fail REGR. vs. 
> 116161

The kernel is hitting a BUG() in gnttab_batch_copy() (see stack trace). This 
seems to
be because GNTTABOP_copy is failing. Looking at the serial log, this seems to 
happen 
time to time on the Arndale (not on the cubietruck) with different version of 
the kernel.

I have reported a similar error last year ([1]), and still have no clue why 
page-table 
translation might fail...

I am going to send a patch adding a bit more debug in the function doing the
translation from the guest PA to the host PA. Hopefully, it might tell us a bit 
more
what's going on.

Cheers,

[1] https://lists.xen.org/archives/html/xen-devel/2016-07/msg02571.htm

Nov 15 05:23:47.715172 [ 2156.529661] ------------[ cut here ]------------

Nov 15 05:24:04.483235 [ 2156.532899] kernel BUG at 
drivers/xen/grant-table.c:770!

Nov 15 05:24:04.491191 [ 2156.538281] Internal error: Oops - BUG: 0 [#1] SMP ARM

Nov 15 05:24:04.491233 [ 2156.543488] Modules linked in: xen_gntalloc 
snd_soc_i2s snd_soc_idma snd_soc_s3c_dma snd_soc_core snd_pcm_dmaengine snd_pcm 
wm8994_regulator snd_timer snd s5p_mfc wm8994 soundcore ac97_bus 
videobuf2_dma_contig videobuf2_memops pwm_samsung videobuf2_v4l2 videobuf2_core 
v4l2_common videodev media s5p_sss usb3503 rtc_s3c dwc3 dwc3_exynos clk_s2mps11 
s5m8767 dw_mmc_exynos dw_mmc_pltfm dw_mmc phy_exynos5250_sata phy_exynos_usb2 
ohci_exynos ehci_exynos phy_exynos5_usbdrd

Nov 15 05:24:04.531197 [ 2156.584721] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 
4.9.20+ #1

Nov 15 05:24:04.539232 [ 2156.590793] Hardware name: SAMSUNG EXYNOS (Flattened 
Device Tree)

Nov 15 05:24:04.547155 [ 2156.596957] task: c1207540 task.stack: c1200000

Nov 15 05:24:04.547179 [ 2156.601564] PC is at gnttab_batch_copy+0xd0/0xe4

Nov 15 05:24:04.555165 [ 2156.606246] LR is at gnttab_batch_copy+0x1c/0xe4

Nov 15 05:24:04.563145 [ 2156.610932] pc : [<c06dba3c>]    lr : [<c06db988>]    
psr: a0000113

Nov 15 05:24:04.563169 [ 2156.610932] sp : c1201d98  ip : deadbeef  fp : 
e160c000

Nov 15 05:24:04.571111 [ 2156.622563] r10: c1201e90  r9 : 00000040  r8 : 
00000040

Nov 15 05:24:04.579105 [ 2156.627858] r7 : e160c000  r6 : 00000001  r5 : 
e1610e00  r4 : e160e6d8

Nov 15 05:24:04.587154 [ 2156.634453] r3 : e160e6d8  r2 : deadbeef  r1 : 
deadbeef  r0 : fffffff2

Nov 15 05:24:04.587189 [ 2156.641054] Flags: NzCv  IRQs on  FIQs on  Mode 
SVC_32  ISA ARM  Segment none

Nov 15 05:24:04.595168 [ 2156.648256] Control: 10c5387d  Table: 7684006a  DAC: 
00000051

Nov 15 05:24:04.603139 [ 2156.654072] Process swapper/0 (pid: 0, stack limit = 
0xc1200220)

Nov 15 05:24:04.611224 [ 2156.660147] Stack: (0xc1201d98 to 0xc1202000)

Nov 15 05:24:04.611279 [ 2156.664574] 1d80:                                     
                  e160e6d8 00000000

Nov 15 05:24:04.619248 [ 2156.672824] 1da0: e1610e00 00000000 e160c000 c08b65a0 
da3cb580 00000000 00000002 c039c1f8

Nov 15 05:24:04.627228 [ 2156.681080] 1dc0: c8c27db0 c039c85c 000f4240 00000000 
c1201df4 e160e6d8 0e29ffbb 000001f6

Nov 15 05:24:04.635141 [ 2156.689314] 1de0: f26f65f2 20000193 00000000 c8c27d48 
00000008 00000000 00000001 0000005f

Nov 15 05:24:04.643124 [ 2156.697561] 1e00: c8c27dfc 60000193 c8c27d48 c090be6c 
000f4240 00000000 00000000 c0372be4

Nov 15 05:24:04.651211 [ 2156.705807] 1e20: c03103bc c058eec4 0002d51e c0752c50 
c8c27d48 0f268479 00000001 e160c020

Nov 15 05:24:04.667186 [ 2156.714053] 1e40: e160c020 00000000 e160c000 00000040 
00000040 c1201e90 192b8000 c08b9124

Nov 15 05:24:04.675174 [ 2156.722298] 1e60: e160c020 00000001 0002d520 0000012c 
c1202d00 c0a5586c 00000008 da3ce740

Nov 15 05:24:04.683182 [ 2156.730545] 1e80: c1116740 c131473e c1204f20 c1204f20 
c1201e90 c1201e90 c1201e98 c1201e98

Nov 15 05:24:04.691270 [ 2156.738791] 1ea0: 00000000 00000000 00000003 c120208c 
c1200000 c1202080 00000100 c1202080

Nov 15 05:24:04.699158 [ 2156.747037] 1ec0: 40000003 c0348760 df003000 c11151a8 
c1201ec8 c131b200 0000000a 0002d51f

Nov 15 05:24:04.707231 [ 2156.755282] 1ee0: c1202d00 00200100 d9808000 c1113e04 
00000000 00000000 00000001 d9808000

Nov 15 05:24:04.715132 [ 2156.763529] 1f00: df003000 c11151a8 c12030a0 c0348b7c 
00000095 c038aa74 c123f3c8 c1203440

Nov 15 05:24:04.723132 [ 2156.771785] 1f20: df00200c c1201f50 df002000 c0301754 
c030928c c0309290 60000013 ffffffff

Nov 15 05:24:04.731168 [ 2156.780021] 1f40: c1201f84 00000000 c1200000 c030d10c 
00000001 00000000 00000001 c031c520

Nov 15 05:24:04.739171 [ 2156.788266] 1f60: c1200000 c1203034 c1203098 00000001 
00000000 00000000 c11151a8 c12030a0

Nov 15 05:24:04.747165 [ 2156.796513] 1f80: 192b8000 c1201fa0 c030928c c0309290 
60000013 ffffffff 00000051 00000000

Nov 15 05:24:04.755233 [ 2156.804759] 1fa0: 00000000 c037d88c c1201fa8 c12387b1 
00000000 ffffffff 00000000 c1000c5c

Nov 15 05:24:04.763183 [ 2156.813005] 1fc0: ffffffff ffffffff 00000000 c100068c 
00000000 c10abe40 c1318ed4 c120301c

Nov 15 05:24:04.771237 [ 2156.821250] 1fe0: c10abe3c c12087e0 6020406a 410fc0f4 
00000000 6020807c 00000000 00000000

Nov 15 05:24:04.779267 [ 2156.829511] [<c06dba3c>] (gnttab_batch_copy) from 
[<c08b65a0>] (xenvif_tx_action+0x80/0x738)

Nov 15 05:24:04.787225 [ 2156.838010] [<c08b65a0>] (xenvif_tx_action) from 
[<c08b9124>] (xenvif_poll+0x28/0x64)

Nov 15 05:24:04.795188 [ 2156.845908] [<c08b9124>] (xenvif_poll) from 
[<c0a5586c>] (net_rx_action+0x1e4/0x2d8)

Nov 15 05:24:04.803191 [ 2156.853717] [<c0a5586c>] (net_rx_action) from 
[<c0348760>] (__do_softirq+0xfc/0x218)

Nov 15 05:24:04.811217 [ 2156.861528] [<c0348760>] (__do_softirq) from 
[<c0348b7c>] (irq_exit+0xe4/0x140)

Nov 15 05:24:04.819157 [ 2156.868908] [<c0348b7c>] (irq_exit) from [<c038aa74>] 
(__handle_domain_irq+0x60/0xb4)

Nov 15 05:24:04.827160 [ 2156.876806] [<c038aa74>] (__handle_domain_irq) from 
[<c0301754>] (gic_handle_irq+0x48/0x8c)

Nov 15 05:24:04.835169 [ 2156.885223] [<c0301754>] (gic_handle_irq) from 
[<c030d10c>] (__irq_svc+0x6c/0x90)

Nov 15 05:24:04.843205 [ 2156.892771] Exception stack(0xc1201f50 to 0xc1201f98)

Nov 15 05:24:04.843240 [ 2156.897894] 1f40:                                     
00000001 00000000 00000001 c031c520

Nov 15 05:24:04.859140 [ 2156.906141] 1f60: c1200000 c1203034 c1203098 00000001 
00000000 00000000 c11151a8 c12030a0

Nov 15 05:24:04.867283 [ 2156.914387] 1f80: 192b8000 c1201fa0 c030928c c0309290 
60000013 ffffffff

Nov 15 05:24:04.867324 [ 2156.921078] [<c030d10c>] (__irq_svc) from 
[<c0309290>] (arch_cpu_idle+0x38/0x3c)

Nov 15 05:24:04.875243 [ 2156.928542] [<c0309290>] (arch_cpu_idle) from 
[<c037d88c>] (cpu_startup_entry+0x194/0x218)

Nov 15 05:24:04.883200 [ 2156.936874] [<c037d88c>] (cpu_startup_entry) from 
[<c1000c5c>] (start_kernel+0x380/0x38c)

Nov 15 05:24:04.891277 [ 2156.945116] Code: e1c432b4 eaffffe0 e7f001f2 e8bd80f8 
(e7f001f2) 

Nov 15 05:24:04.899168 [ 2156.951298] ---[ end trace 766604e7ecb29bdc ]---

Nov 15 05:24:04.907166 [ 2156.955961] Kernel panic - not syncing: Fatal 
exception in interrupt

Nov 15 05:24:04.915143 [ 2156.962415] CPU1: stopping

Nov 15 05:24:04.915174 [ 2156.965166] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G  
    D         4.9.20+ #1

Nov 15 05:24:04.923290 [ 2156.972453] Hardware name: SAMSUNG EXYNOS (Flattened 
Device Tree)

Nov 15 05:24:04.931162 [ 2156.978631] [<c0310f94>] (unwind_backtrace) from 
[<c030c574>] (show_stack+0x10/0x14)

Nov 15 05:24:04.939192 [ 2156.986436] [<c030c574>] (show_stack) from 
[<c0591068>] (dump_stack+0x98/0xac)

Nov 15 05:24:04.939219 [ 2156.993729] [<c0591068>] (dump_stack) from 
[<c030f688>] (handle_IPI+0x174/0x194)

Nov 15 05:24:04.947171 [ 2157.001192] [<c030f688>] (handle_IPI) from 
[<c0301794>] (gic_handle_irq+0x88/0x8c)

Nov 15 05:24:04.955151 [ 2157.008829] [<c0301794>] (gic_handle_irq) from 
[<c030d10c>] (__irq_svc+0x6c/0x90)

Nov 15 05:24:04.963147 [ 2157.016376] Exception stack(0xd98dbf88 to 0xd98dbfd0)

Nov 15 05:24:04.971150 [ 2157.021597] bf80:                   00000001 00000000 
00000001 c031c520 d98da000 c1203034

Nov 15 05:24:04.979206 [ 2157.029850] bfa0: c1203098 00000002 00000000 00000000 
c11151a8 c12030a0 192c6000 d98dbfd8

Nov 15 05:24:04.987142 [ 2157.038093] bfc0: c030928c c0309290 600f0013 ffffffff

Nov 15 05:24:04.995142 [ 2157.043118] [<c030d10c>] (__irq_svc) from 
[<c0309290>] (arch_cpu_idle+0x38/0x3c)

Nov 15 05:24:05.003148 [ 2157.050584] [<c0309290>] (arch_cpu_idle) from 
[<c037d88c>] (cpu_startup_entry+0x194/0x218)

Nov 15 05:24:05.011218 [ 2157.058913] [<c037d88c>] (cpu_startup_entry) from 
[<60301c4c>] (0x60301c4c)

Nov 15 05:24:05.011257 [ 2157.066095] ---[ end Kernel panic - not syncing: 
Fatal exception in interrupt


-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel