Re: [Xen-devel] 4.11.0 RC1 panic

[Manuel Bouyer <bouyer@xxxxxxxxxxxxxxx>]

On Wed, Apr 25, 2018 at 09:28:03AM -0600, Jan Beulich wrote:
> >>> On 25.04.18 at 16:42, <bouyer@xxxxxxxxxxxxxxx> wrote:
> > On Wed, Apr 25, 2018 at 12:42:42PM +0200, Manuel Bouyer wrote:
> >> > Without line numbers associated with at least the top stack trace entry
> >> > I can only guess what it might be - could you give the patch below a try?
> >> > (This may not be the final patch, as I'm afraid there may be some race
> >> > here, but I'd have to work this out later.)
> >> 
> >> Yes, this works. thanks !
> >> I'll now put this version on the NetBSD testbed I'm running.
> >> This should put some pressure on it.
> > 
> > Running NetBSD tests in several guests I got:
> > (XEN) 
> > (XEN) ****************************************
> > (XEN) Panic on CPU 1:
> > (XEN) Assertion 'oc > 0' failed at mm.c:628
> > (XEN) ****************************************
> > (see attached file for complete report).
> 
> Do you know what exactly the guest was doing at that time?

Unfortunably no. It was running the NetBSD test benchs, but as this
is automated I don't even know what version of NetBSD was running
in the guests.
BTW there doesn't seem to be a domain number in the panic message ...

> IOW do
> you have any information on how to repro (preferably without having
> to run NetBSD)?

Unfortunably no, and it's not reliably reproductible either.
A cron job starts running the tests for available builds daily,
and the panic occurs once in a while.

You may be able to reproduce it with a linux dom0:
install anita from http://www.gson.org/netbsd/anita/download/
this is a set of python script; so you should be able to
extract the tar.gz and run the anita script in there.

Then run:
./anita --test-timeout 14400 --vmm xl --vmm-args vcpus=4 --disk-size 2G 
--memory-size 256M test 
http://ftp.fr.netbsd.org/pub/NetBSD-daily/HEAD/201804210730Z/amd64/
you will have to adjust the URLs: these are daily builds, and older versions
are deleted when newer ones are build. You can also use other branches
instead of HEAD.

Eventually Xen will panic (but only once in a while).


> Did these failures start occurring recently (your
> mention of 4.8 seems to suggest otherwise)?

Looking at the server's log, the first time I've seen them was with
Xen 4.6.6, with patches up to XSA244. Before that it was running 4.6.5
with patch for XSA-212. It looks like the ASSERT() was added as part of
XSA240.

Then I upgraded to Xen 4.8.x (also with the security patches) but this
didn't fix the problem. I still had it with 4.8.3, and now with 4.11 too
(I didn't try anything else between 4.8 and 4.11)

-- 
Manuel Bouyer <bouyer@xxxxxxxxxxxxxxx>
     NetBSD: 26 ans d'experience feront toujours la difference
--

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel