Re: [Xen-devel] 100% reliable Oops on xen 4.0.1

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Tue, 2012-08-14 at 01:03 +0100, Peter Moody wrote:
> This seems to be some combination of Xen and the audit subsystem, but
> the attached program crashes my machine 100% of the time.
> 
> steps to reproduce the crash:
> 
>  *  1) compile with gcc -m32
>  *  2) start auditd, install any rule (I've only tested syscall
> auditing, but any syscall seems to work).
>  *     /etc/init.d/auditd start ; auditctl -D ; auditctl -a
> exit,always -F arch=64 -S chmod
>  *  3) run'n wait (this only loops twice for me before dying)
>  *     ./a.out
>  *  4) bask in instantaneous kernel oops.
> 
> here's xm info from dom0
> 
> [xen2.atl] root@gntb1:~# xm info
> host                   : gntb1.atl.corp.google.com
> release                : 3.2.13-ganeti-rx6-xen0
> version                : #1 SMP Thu Jun 7 12:59:40 CEST 2012
> machine                : x86_64
> nr_cpus                : 12
> nr_nodes               : 2
> cores_per_socket       : 6
> threads_per_core       : 1
> cpu_mhz                : 2660
> hw_caps                :
> bfebfbff:2c100800:00000000:00001f40:029ee3ff:00000000:00000001:00000000
> virt_caps              : hvm
> total_memory           : 32755
> free_memory            : 22665
> node_to_cpu            : node0:0,2,4,6,8,10
>                          node1:1,3,5,7,9,11
> node_to_memory         : node0:13083
>                          node1:9582
> node_to_dma32_mem      : node0:0
>                          node1:3235
> max_node_id            : 1
> xen_major              : 4
> xen_minor              : 0
> xen_extra              : .1
> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
> hvm-3.0-x86_32p hvm-3.0-x86_64
> xen_scheduler          : credit
> xen_pagesize           : 4096
> platform_params        : virt_start=0xffff800000000000
> xen_changeset          : unavailable
> xen_commandline        : placeholder dom0_mem=1024M loglvl=all
> com1=115200,8n1 console=com1 iommu=0
> cc_compiler            : gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
> cc_compile_by          : pmacedo
> cc_compile_domain      : google.com
> cc_compile_date        : Wed Mar 16 15:24:06 UTC 2011
> xend_config_format     : 4
> 
> I'm not sure what you need from the domU. It's running 2.6.38.8 (but
> I've seen this bug all the way up to 3.5.0-rc7, the latest I've
> tested). It's a fairly beefy setup, 32G memory and 6 cpus.
> 
> I suspect xen as opposed to auditd because:
> 
>  a) this only happens on our xen machines (though not all of them)
>  b) one of my stack traces started with
> 
> [172577.560441]  [<ffffffff810065ad>] ? xen_force_evtchn_callback+0xd/0x10

This is likely to be a coincidence IMHO since this function forces a
call to the hypervisor to trigger the (re)injection of any pending
interrupts (typically after reenabling interrupts), so it is not unusual
for it to be at the bottom of any stack trace which happens in interrupt
context.

The example stack trace in crasher.c doesn't involve Xen -- can you post
any examples of ones which do.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel