[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] [OSSTEST PATCH v2] sg-run-job: Coalesce a couple of repetitions
On Mon, 2015-11-30 at 16:49 +0000, Ian Jackson wrote: > Fold `guest-localmigrate.2' into `guest-localmigrate/x10' and move > `guest-start.2' to after `guest-start.repeat' (reversing the contents > of the latter so that the start comes before the stop). > (guest-start.2 is still necessary because the start/stop test leaves > the guest stopped, whereas the subsequent destroy test ought happen > with the guest running.) > > This change will allow the heisenbug compensator to see more of these > failures as the same failures. > > The overall effect includes a reduction of the number of localhost > migrations from 11 to 10, but this is better than leaving a misleading > testid containing the string `x10' (or changing the testid). > > It is best to fold this way, keeping the testid of the step which > previously had most of the regressions, because: the alternative, > keeping the testid of the low-repetition step, would allow osstest to > use previous lucky passes of the low-repetition step to justify > current failures of the now-high-repetition step. > > To check that the effect of the patch is as intended, I ran a before > and after run with OSSTEST_SIMULATE=1, and (a) collected and sedded > and diffed the sg-run-job transcripts and (b) looked in the db. > > I also ran a real test (65261 in the Xen Project test lab) with a very > similar version, which passed, and will re-run that before pushing. > > (a): > > Â c&p transcripts from mg-execute-flight email reports > Â perl -i~ -pe 's/\b(38371|38370|65261|38395|38397)\b/FLIGHT/; s/^2015- > 11-\d\d \S+ /TIME /' [tu] > Â diff -u [tu] >v > Â grep starting v > > Â=> > > ÂTIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-build- > checkÂÂbuild-check(1) > ÂTIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest- > saverestore host debian guest-saverestore.2 > -TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest- > localmigrate host debian guest-localmigrate.2 > ÂTIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest- > localmigrate x10 host debian guest-localmigrate/x10 > ÂTIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest- > stop host debian guest-stop > +TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts- > repeat-test 10 ts-guest-start host debian {;} ts-guest-stop host debian > guest-start/debian.repeat > ÂTIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest- > start host debian guest-start.2 > -TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts- > repeat-test 10 ts-guest-stop host debian {;} ts-guest-start host debian > guest-start/debian.repeat > ÂTIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest- > destroy host debian guest-destroy > ÂTIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-leak- > check check host leak-check/check > -TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-logs- > capture host capture-logs(24) > +TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-logs- > capture host capture-logs(23) > > (b) > > osstestdb=> select * from (select job,stepno,step,status,testid from > steps where flight=38370) before full outer join (select > job,stepno,step,status,testid from steps where flight=38400) after using > (testid) order by coalesce(before.stepno, after.stepno); > ÂÂÂÂÂÂÂÂÂÂtestidÂÂÂÂÂÂÂÂÂÂÂ|ÂÂÂÂÂÂÂÂjobÂÂÂÂÂÂÂÂÂ| stepno > |ÂÂÂÂÂÂÂÂÂÂÂÂÂstepÂÂÂÂÂÂÂÂÂÂÂÂÂ| status |ÂÂÂÂÂÂÂÂjobÂÂÂÂÂÂÂÂÂ| stepno > |ÂÂÂÂÂÂÂÂÂÂÂÂÂstepÂÂÂÂÂÂÂÂÂÂÂÂÂ| status > ---------------------------+--------------------+--------+--------------- > ---------------+--------+--------------------+--------+---------------- > --------------+-------- > Âbuild-check(1)ÂÂÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ1 | ts-build- > checkÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ1 | ts-build- > checkÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âhosts-allocateÂÂÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ2 | ts-hosts- > allocateÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ2 | ts-hosts- > allocateÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âhost-install(3)ÂÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ3 | ts-host- > install-twiceÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ3 | ts-host- > install-twiceÂÂÂÂÂÂÂÂ| pass > Âhost-ping-check-nativeÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ4 | ts-host-ping- > checkÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ4 | ts-host-ping- > checkÂÂÂÂÂÂÂÂÂÂÂ| pass > Âxen-installÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ5 | ts-xen- > installÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ5 | ts-xen- > installÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âxen-bootÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ6 | ts-host- > rebootÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ6 | ts-host- > rebootÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âhost-ping-check-xenÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ7 | ts-host-ping- > checkÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ7 | ts-host-ping- > checkÂÂÂÂÂÂÂÂÂÂÂ| pass > Âleak-check/basis(8)ÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ8 | ts-leak- > checkÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ8 | ts-leak- > checkÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âdebian-installÂÂÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ9 | ts-debian- > installÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂÂ9 | ts-debian- > installÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âdebian-fixupÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ10 | ts-debian- > fixupÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ10 | ts-debian- > fixupÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âguest-startÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ11 | ts-guest- > startÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ11 | ts-guest- > startÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âmigrate-support-checkÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ12 | ts-migrate- > support-checkÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ12 | ts-migrate- > support-checkÂÂÂÂÂ| pass > Âsaverestore-support-check | test-amd64-i386-xl |ÂÂÂÂÂ13 | ts- > saverestore-support-check | passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ13 | ts- > saverestore-support-check | pass > Âguest-saverestoreÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ14 | ts-guest- > saverestoreÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ14 | ts-guest- > saverestoreÂÂÂÂÂÂÂÂÂ| pass > Âguest-localmigrateÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ15 | ts-guest- > localmigrateÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ15 | ts-guest- > localmigrateÂÂÂÂÂÂÂÂ| pass > Âguest-saverestore.2ÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ16 | ts-guest- > saverestoreÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ16 | ts-guest- > saverestoreÂÂÂÂÂÂÂÂÂ| pass > Âguest-localmigrate.2ÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ17 | ts-guest- > localmigrateÂÂÂÂÂÂÂÂ| > passÂÂÂ|ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ|ÂÂÂÂÂÂÂÂ|ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| > Âguest-localmigrate/x10ÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ18 | ts-guest- > localmigrateÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ17 | ts-guest- > localmigrateÂÂÂÂÂÂÂÂ| pass > Âguest-stopÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ19 | ts-guest- > stopÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ18 | ts-guest- > stopÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âguest-start.2ÂÂÂÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ20 | ts-guest- > startÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ20 | ts-guest- > startÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âguest-start/debian.repeat | test-amd64-i386-xl |ÂÂÂÂÂ21 | ts-repeat- > testÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ19 | ts-repeat- > testÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âguest-destroyÂÂÂÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ22 | ts-guest- > destroyÂÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ21 | ts-guest- > destroyÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âleak-check/checkÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ23 | ts-leak- > checkÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| passÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ22 | ts-leak- > checkÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âcapture- > logs(23)ÂÂÂÂÂÂÂÂÂÂ|ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ|ÂÂÂÂÂÂÂÂ|ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ > ÂÂÂÂÂÂ|ÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ23 | ts-logs- > captureÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| pass > Âcapture-logs(24)ÂÂÂÂÂÂÂÂÂÂ| test-amd64-i386-xl |ÂÂÂÂÂ24 | ts-logs- > captureÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| > passÂÂÂ|ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ|ÂÂÂÂÂÂÂÂ|ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ| > (25 rows) > > osstestdb=> > > Signed-off-by: Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx> Acked-by: Ian Campbell <ian.campbell@xxxxxxxxxx> _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |