Re: [Xen-devel] [xen-4.6-testing test] 65112: regressions - FAIL

[Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>]

Ian Campbell writes ("Re: [Xen-devel] [xen-4.6-testing test] 65112: regressions 
- FAIL"):
> On Fri, 2015-11-27 at 12:02 +0000, Ian Jackson wrote:
> > As explained below, in 65112 this step did not run because the earlier
> > step `guest-localmigrate' failed:
> >   http://logs.test-lab.xenproject.org/osstest/logs/65112/test-amd64-
> > amd64-xl-qemut-stubdom-debianhvm-amd64-xsm/info.html
> 
> Would it be possible to arrange for "blocked" to appear somewhere in the
> results for the job? e.g. "blocked fail in XXX REGR. vs. YYY". README.email
> says "The results normally start with the result in this flight" and I
> think this would be in keeping with that.

But it might not be true that it was blocked.  Maybe the version of
osstest used didn't have that step at all, for example.

The best you could say would be something like
  "not run; fail in XXX REGR. vs. YYY"
but that poses more questions than it answers.

> Otherwise I think people naturally tend to just read the "and are blocking"
> section and forget to consider that non-blocking stuff further down may
> have (tolerably) failed but then blocking something else which is then
> blocking the push.

Perhaps sg-report-flight could, if there are any blockages of the form
`fail in XXX REGR. vs YYY', add a note below the blockage section,
saying something like `XXX examined since needed to justify other
failures, see below'.

I'm a bit reluctant to suggest this because it is, essentially,
boilerplate - it would always say the same thing about any `fail in
XXX' - and filling reports like this with boilerplate isn't always a
good idea.

> > The fact that we have both `guest-localmigrate' and
> > `guest-localmigrate/x10' isn't ideal because it hides from the
> > heisenbug compensator that these are actually the same underlying
> > test.  Maybe it is time now to rename `guest-localmigrate/x10' to
> > `guest-localmigrate' and abolish the latter.
> 
> I think this would be a good idea.

I'll send a patch.

> > From 987dd088192f9f94c59beeddc073cecaad76a24e Mon Sep 17 00:00:00 2001
> > From: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
> > Date: Fri, 27 Nov 2015 11:36:05 +0000
> > Subject: [OSSTEST PATCH] README.email: Document `fail in 58948 REGR. vs.
> >  63449'
> > 
> > Signed-off-by: Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>
> 
> Acked-by: Ian Campbell <ian.campbell@xxxxxxxxxx>

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel