Re: [Xen-devel] Linux 4.8-rc8 with Xen for-linus-4.9 branch: dom0 crashes on boot

[Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx>]

On 10/02/2016 08:26 AM, Sander Eikelenboom wrote:
> On 2016-10-02 14:08, Andrew Cooper wrote:
>> On 02/10/2016 12:46, Sander Eikelenboom wrote:
>>> Hi All,
>>>
>>> Since the new merge window is emerging I took the liberty of testing a
>>> linux 4.8-rc8 tree with
>>> the Xen for-linus-4.9 branch pulled on top.
>>> Unfortunately this crashes dom0 early in boot under Xen.
>>> On bare-metal the same kernel boots fine.
>>> Under Xen a linux 4.8-rc8 kernel without the Xen branch pulled on top,
>>> also boots fine.
>>
>> So this looks to be a regression in the Xen for-linux-4.9 branch.
>>
>>>
>>> Hypervisor is a recentish xen-unstable build.
>>>
>>> The serial log is below.
>>> <snip>
>>> (XEN) [2016-10-02 11:31:53.106] Scrubbing Free RAM on 1 nodes using 6
>>> CPUs
>>> (XEN) [2016-10-02 11:31:53.217] .............................done.
>>> (XEN) [2016-10-02 11:31:56.242] Initial low memory virq threshold set
>>> at 0x4000 pages.
>>> (XEN) [2016-10-02 11:31:56.260] Std. Loglevel: All
>>> (XEN) [2016-10-02 11:31:56.277] Guest Loglevel: All
>>> (XEN) [2016-10-02 11:31:56.295] Xen is relinquishing VGA console.
>>> (XEN) [2016-10-02 11:31:56.396] *** Serial input -> DOM0 (type
>>> 'CTRL-a' three times to switch input to Xen)
>>> (XEN) [2016-10-02 11:31:56.396] Freed 308kB init memory
>>> (XEN) [2016-10-02 11:31:56.397] d0v0: unhandled page fault (ec=0000)
>>> (XEN) [2016-10-02 11:31:56.397] Pagetable walk from 0000000000000001:
>>> (XEN) [2016-10-02 11:31:56.397]  L4[0x000] = 0000000000000000
>>> ffffffffffffffff
>>> (XEN) [2016-10-02 11:31:56.397] domain_crash_sync called from entry.S:
>>> fault at ffff82d080244960 entry.o#create_bounce_frame+0x145/0x154
>>> (XEN) [2016-10-02 11:31:56.397] Domain 0 (vcpu#0) crashed on cpu#0:
>>> (XEN) [2016-10-02 11:31:56.397] ----[ Xen-4.8-unstable  x86_64
>>> debug=y   Not tainted ]----
>>> (XEN) [2016-10-02 11:31:56.397] CPU:    0
>>> (XEN) [2016-10-02 11:31:56.397] RIP:    e033:[<ffffffff8101fdb9>]
>>> (XEN) [2016-10-02 11:31:56.397] RFLAGS: 0000000000000286   EM: 1
>>> CONTEXT: pv guest (d0v0)
>>> (XEN) [2016-10-02 11:31:56.397] rax: 0000000000000000   rbx:
>>> ffffffff82248bb0   rcx: ffffffff8101bc10
>>> (XEN) [2016-10-02 11:31:56.397] rdx: 0000000000000001   rsi:
>>> ffffffff81f0aa50   rdi: ffffffff82248bb0
>>> (XEN) [2016-10-02 11:31:56.397] rbp: ffffffff82203e50   rsp:
>>> ffffffff82203dc0   r8:  ffffffff8101b550
>>> (XEN) [2016-10-02 11:31:56.397] r9:  0000000000000000   r10:
>>> 0000000000000000   r11: 0000000080802001
>>> (XEN) [2016-10-02 11:31:56.397] r12: 0000000000000000   r13:
>>> 0000000000000000   r14: ffffffff82215580
>>> (XEN) [2016-10-02 11:31:56.397] r15: 0000000000000000   cr0:
>>> 000000008005003b   cr4: 00000000000006e0
>>> (XEN) [2016-10-02 11:31:56.397] cr3: 000000054a601000   cr2:
>>> 0000000000000001
>>> (XEN) [2016-10-02 11:31:56.397] ds: 0000   es: 0000   fs: 0000   gs:
>>> 0000   ss: e02b   cs: e033
>>> (XEN) [2016-10-02 11:31:56.397] Guest stack trace from
>>> rsp=ffffffff82203dc0:
>>> (XEN) [2016-10-02 11:31:56.397]    ffffffff8101bc10 0000000080802001
>>> 0000000000000000 ffffffff8101fdb9
>>> (XEN) [2016-10-02 11:31:56.397]    000000010000e030 0000000000010086
>>> ffffffff82203e00 000000000000e02b
>>> (XEN) [2016-10-02 11:31:56.397]    ffffffff82203e50 ffffffff8101fcb5
>>> 0000000080802001 0000000000000000
>>> (XEN) [2016-10-02 11:31:56.397]    0000000000000000 ffffffff8101b550
>>> ffffffff82248bb0 ffffffff81f0aa50
>>> (XEN) [2016-10-02 11:31:56.397]    0000000000000001 ffffffff8101bc10
>>> ffffffff82203eb8 ffffffff81b7e9f4
>>> (XEN) [2016-10-02 11:31:56.397]    0000000000000000 ffffffff82203ea8
>>> 0000000080802001 0000000000000004
>>> (XEN) [2016-10-02 11:31:56.397]    ffffffff8101baa2 ffffffff82203f40
>>> ffffffff82248bb0 0000000000000000
>>> (XEN) [2016-10-02 11:31:56.397]    0000000000000000 ffffffff8101bc10
>>> 0000000000000000 ffffffff82203ed0
>>> (XEN) [2016-10-02 11:31:56.397]    ffffffff81b7ed45 0000000000000013
>>> ffffffff82203ee0 ffffffff810cc127
>>> (XEN) [2016-10-02 11:31:56.397]    ffffffff82203f28 ffffffff810ccdab
>>> ffffffff81f0aa50 ffffffff8101b550
>>> (XEN) [2016-10-02 11:31:56.397]    ffffffff82203f60 ffffffff82203f5c
>>> 0000000000000000 0000000000000000
>>> (XEN) [2016-10-02 11:31:56.397]    0000000000000000 ffffffff82203f40
>>> ffffffff810c83be ffffffff82600000
>>> (XEN) [2016-10-02 11:31:56.397]    ffffffff82203ff8 ffffffff8232946a
>>> 00100fa000000000 8080200100060800
>>> (XEN) [2016-10-02 11:31:56.397]    000000001789c3f5 0000000000000000
>>> 0000000000000000 0000000000000000
>>> (XEN) [2016-10-02 11:31:56.397]    0000000000000000 0000000000000000
>>> 0000000000000000 0000000000000000
>>> (XEN) [2016-10-02 11:31:56.397]    0000000000000000 0000000000000000
>>> 0000000000000000 0000000000000000
>>> (XEN) [2016-10-02 11:31:56.397]    0000000000000000 0000000000000000
>>> 0000000000000000 0000000000000000
>>> (XEN) [2016-10-02 11:31:56.397]    0000000000000000 0000000000000000
>>> 0000000000000000 0000000000000000
>>> (XEN) [2016-10-02 11:31:56.397]    0f00000060c0c748 ccccccccccccc305
>>> cccccccccccccccc cccccccccccccccc
>>> (XEN) [2016-10-02 11:31:56.397]    cccccccccccccccc cccccccccccccccc
>>> cccccccccccccccc cccccccccccccccc
>>> (XEN) [2016-10-02 11:31:56.397] Hardware Dom0 crashed: rebooting
>>> machine in 5 seconds.
>>
>> Something in Linux at ffffffff8101fdb9 followed a NULL pointer.  Can you
>> see what it was with the linux debug symbols?
>>
>> ~Andrew
>
> Sure thing:
> addr2line -e vmlinux-4.8.0-rc8-20161002-linus-xennext+ ffffffff8101fdb9
> /usr/src/new/linux-linus/arch/x86/xen/irq.c:34
>
> asmlinkage __visible unsigned long xen_save_fl(void)
> {
>         struct vcpu_info *vcpu;
>         unsigned long flags;
>
>         vcpu = this_cpu_read(xen_vcpu);
>
>         /* flag has opposite sense of mask */
>         flags = !vcpu->evtchn_upcall_mask;   <== WHICH IS HERE
>
>         /* convert to IF type flag
>            -0 -> 0x00000000
>            -1 -> 0xffffffff
>         */
>         return (-flags) & X86_EFLAGS_IF;
> }


Can you post your .config file?

Our tests seem to have run fine yesterday with those branches, until
apparently electrical problem shut down the whole test farm this morning.

-boris


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel