Re: [Xen-devel] [xen-4.8-testing test] 109585: regressions - FAIL

[Ian Jackson <ian.jackson@xxxxxxxxxxxxx>]

Andrew Cooper writes ("Re: [Xen-devel] [xen-4.8-testing test] 109585: 
regressions - FAIL"):
> These are not actual regressions.
> 
> test-hvm64-lbr-tsx-vmentry is generic test which runs on all hardware,
> but it triggers a host-specific failure mode on Intel Haswell processors
> and later.
...
> On Xen 4.8, this test will reliably fail on all Intel Haswell/Broadwell
> hardware, and reliably pass on all other hardware.
> 
> The reason it is flagged as a regression is because OSSTest is tracking
> regressions by xtf-*-{1...5} results, but scheduling these jobs on any
> hardware.  What has happened is that these jobs previously ran on AMD or
> older Intel hardware, and have now been run on Haswell/Broadwell.

Here is what I wrote on IRC about this.

15:19 <Diziet> osstest doesn't do host-specific regression tracking,
               and because these individual steps don't cause the
               whole job to fail the job isn't host-sticky, so these
               things will keep showing up as new regressions.

15:19 <Diziet> (And anyway we wouldn't want the job to become
               host-stuck or we would only ever run the xtf tests on
               4.8 on those hosts.)

15:21 <Diziet> One easy thing to do would be to manually mark those
               two tests as allowable regressions for 4.8, but of
               course that wouldn't spot if they started to fail on
               unaffected hosts (but do we care about that?)

15:24 <Diziet> Another approach might be to change the set of xtf jobs so that 
               they are explicitly pinned to microarchitectures.
15:24 <andyhhp> +10

I think this is doable.  Let's chat further on IRC and make a plan.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel