Re: [Xen-devel] [qemu-upstream-unstable test] 12436: regressions - FAIL

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Tue, 2012-03-27 at 16:54 +0100, Ian Jackson wrote:
> Ian Campbell writes ("Re: [Xen-devel] [qemu-upstream-unstable test] 12436: 
> regressions - FAIL"):
> > On Mon, 2012-03-26 at 06:21 +0100, xen.org wrote:
> > > flight 12436 qemu-upstream-unstable real [real]
> > > http://www.chiark.greenend.org.uk/~xensrcts/logs/12436/
> > > 
> > > Regressions :-(
> > > 
> > > Tests which did not succeed and are blocking,
> > > including tests which could not be run:
> > >  test-amd64-amd64-xl-win       7 windows-install           fail REGR. vs. 
> > > 11890
> > 
> > Is anyone looking into this failure?
> 
> NAFAIK.
> 
> This test failure is not related to the single commit made to the
> qemu-upstream-unstable tree, since this test doesn't actually even use
> qemu-upstream-unstable.  (It's a bug in the test schedule generator
> that it even considers this test relevant; it should not run it.)

So actually the upstream qemu is actually passing its testing fairly
consistently? Assuming some of the other tests do actually test it.

> I have taken a look at the logs.  I think it is a genuine failure
> showing up a real bug, but the bug is in xen-unstable or
> qemu-xen-unstable or Jeremy's Linux 2.6.32:

Do we see run this sequence in the flights which are supposed to be
testing those or only here?

If we do run it elsewhere then we wouldn't be papering over the issue by
removing it from this test (where it doesn't belong).

> The test system decided in flight 12436 that the install had failed
> because the guest did not, within the timeout, start listening on the
> expected tcp port (which is set up by a guest agent whose installation
> and startup is built into the image I'm using).  The guest screenshot,
> taken after the timeout via vnc, is a black screen.  The guest
> rebooted itself 18 minutes prior to the timeout; it had done a number
> of reboots previously (as expected).  The timeout is 7000 seconds
> (1h56m40s).
> 
> I have searched the database for this test being run on this host
> lake-frog (or its twin, fire-frog) with xen-unstable.  The most recent
> such run was in an adhoc test I ran (flight 11623) for which the main
> logs have expired.  The database (which does not expire) does record
> that the whole ts-windows-install step succeeded (on lake-frog) and
> took 3660 seconds[1].
> 
> This test has also been run on frogs with xen-4.0-testing but AFAICT
> it always fails there, in a different manner, regardless of which
> host, so this is probably not helpful.
> 
> The most recent test run for xen-unstable ran this test successfully
> on moss-bug, where it took 6122s.
> 
> So it seems likely that the problem is that the HVM Windows
> installation on lake-frog takes "too long".  lake-frog is unusual
> amongst my test machines; it's much larger.  It's an AMD machine with
> 32G of RAM, 24 cores over 2 sockets.  Naively it might be expected to
> be faster but perhaps the problem is that it's too NUMA.
> 
> Regardless, though, I think 6000-odd seconds is indeed too long if
> this step previously took 3660 seconds.

Right.

> 
> Ian.
> 
> [1] This 3660 seconds includes a number of activities which are not
> included in the 7000 second timeout, such as the installation of rsync
> on the test host, copying of the Windows install ISO image onto the
> test host, and so forth.



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel