[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [OSSTEST PATCH] sg-run-job: Coalesce a couple of repetitions



Fold `guest-localmigrate.2' into `guest-localmigrate/x10' and
`guest-start.2' into `guest-start.repeat'.  In each case increase the
iteration count of the 2nd test by one so that the total number of
reps remains constant.

This will allow the heisenbug compensator to see more of these
failures as the same failures.

The resulting testid `guest-localmigrate/x10' becomes a bit misleading
in that it now does 11 repetitions.  But we don't want to change the
testid.

It is best to fold this way, keeping the testid of the step which
previously had most of the regressions, because: the alternative,
keeping the testid of the low-repetition step, would allow osstest to
use previous lucky passes of the low-repetition step to justify
current failures of the now-high-repetition step.

To check that the effect of the patch is as intended, I ran a before
and after run with OSSTEST_SIMULATE=1, and (a) collected and sedded
and diffed the sg-run-job transcripts and (b) looked in the db.

(a):

  c&p transcripts from mg-execute-flight email reports
  perl -i~ -pe 's/\b(38371|38370)\b/FLIGHT/; s/^2015-11-27 \S+ /TIME /' [tu]
  diff -u [tu] >v
  grep starting v

 =>

  TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-build-check 
 build-check(1)
  TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl 
ts-guest-saverestore host debian guest-saverestore.2
 -TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl 
ts-guest-localmigrate host debian guest-localmigrate.2
 -TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl 
ts-guest-localmigrate x10 host debian guest-localmigrate/x10
 +TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl 
ts-guest-localmigrate x11 host debian guest-localmigrate/x10
  TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest-stop 
host debian guest-stop
 -TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-guest-start 
host debian guest-start.2
 -TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-repeat-test 
10 ts-guest-stop host debian {;} ts-guest-start host debian 
guest-start/debian.repeat
 +TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-repeat-test 
11 ts-guest-stop host debian {;} ts-guest-start host debian 
guest-start/debian.repeat
  TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl 
ts-guest-destroy host debian guest-destroy
  TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl ts-leak-check 
check host leak-check/check
 -TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl 
ts-logs-capture host capture-logs(24)
 +TIME Z [test-amd64-i386-xl] starting FLIGHT.test-amd64-i386-xl 
ts-logs-capture host capture-logs(22)

(b)

osstestdb=> select * from (select job,stepno,step,status,testid from steps 
where flight=38370) before full outer join (select 
job,stepno,step,status,testid from steps where flight=38371) after using 
(testid) order by coalesce(before.stepno, after.stepno);
          testid           |        job         | stepno |             step     
        | status |        job         | stepno |             step             | 
status
---------------------------+--------------------+--------+------------------------------+--------+--------------------+--------+------------------------------+--------
 build-check(1)            | test-amd64-i386-xl |      1 | ts-build-check       
        | pass   | test-amd64-i386-xl |      1 | ts-build-check               | 
pass
 hosts-allocate            | test-amd64-i386-xl |      2 | ts-hosts-allocate    
        | pass   | test-amd64-i386-xl |      2 | ts-hosts-allocate            | 
pass
 host-install(3)           | test-amd64-i386-xl |      3 | 
ts-host-install-twice        | pass   | test-amd64-i386-xl |      3 | 
ts-host-install-twice        | pass
 host-ping-check-native    | test-amd64-i386-xl |      4 | ts-host-ping-check   
        | pass   | test-amd64-i386-xl |      4 | ts-host-ping-check           | 
pass
 xen-install               | test-amd64-i386-xl |      5 | ts-xen-install       
        | pass   | test-amd64-i386-xl |      5 | ts-xen-install               | 
pass
 xen-boot                  | test-amd64-i386-xl |      6 | ts-host-reboot       
        | pass   | test-amd64-i386-xl |      6 | ts-host-reboot               | 
pass
 host-ping-check-xen       | test-amd64-i386-xl |      7 | ts-host-ping-check   
        | pass   | test-amd64-i386-xl |      7 | ts-host-ping-check           | 
pass
 leak-check/basis(8)       | test-amd64-i386-xl |      8 | ts-leak-check        
        | pass   | test-amd64-i386-xl |      8 | ts-leak-check                | 
pass
 debian-install            | test-amd64-i386-xl |      9 | ts-debian-install    
        | pass   | test-amd64-i386-xl |      9 | ts-debian-install            | 
pass
 debian-fixup              | test-amd64-i386-xl |     10 | ts-debian-fixup      
        | pass   | test-amd64-i386-xl |     10 | ts-debian-fixup              | 
pass
 guest-start               | test-amd64-i386-xl |     11 | ts-guest-start       
        | pass   | test-amd64-i386-xl |     11 | ts-guest-start               | 
pass
 migrate-support-check     | test-amd64-i386-xl |     12 | 
ts-migrate-support-check     | pass   | test-amd64-i386-xl |     12 | 
ts-migrate-support-check     | pass
 saverestore-support-check | test-amd64-i386-xl |     13 | 
ts-saverestore-support-check | pass   | test-amd64-i386-xl |     13 | 
ts-saverestore-support-check | pass
 guest-saverestore         | test-amd64-i386-xl |     14 | ts-guest-saverestore 
        | pass   | test-amd64-i386-xl |     14 | ts-guest-saverestore         | 
pass
 guest-localmigrate        | test-amd64-i386-xl |     15 | 
ts-guest-localmigrate        | pass   | test-amd64-i386-xl |     15 | 
ts-guest-localmigrate        | pass
 guest-saverestore.2       | test-amd64-i386-xl |     16 | ts-guest-saverestore 
        | pass   | test-amd64-i386-xl |     16 | ts-guest-saverestore         | 
pass
 guest-localmigrate.2      | test-amd64-i386-xl |     17 | 
ts-guest-localmigrate        | pass   |                    |        |           
                   |
 guest-localmigrate/x10    | test-amd64-i386-xl |     18 | 
ts-guest-localmigrate        | pass   | test-amd64-i386-xl |     17 | 
ts-guest-localmigrate        | pass
 guest-stop                | test-amd64-i386-xl |     19 | ts-guest-stop        
        | pass   | test-amd64-i386-xl |     18 | ts-guest-stop                | 
pass
 guest-start.2             | test-amd64-i386-xl |     20 | ts-guest-start       
        | pass   |                    |        |                              |
 guest-start/debian.repeat | test-amd64-i386-xl |     21 | ts-repeat-test       
        | pass   | test-amd64-i386-xl |     19 | ts-repeat-test               | 
pass
 capture-logs(22)          |                    |        |                      
        |        | test-amd64-i386-xl |     22 | ts-logs-capture              | 
pass
 guest-destroy             | test-amd64-i386-xl |     22 | ts-guest-destroy     
        | pass   | test-amd64-i386-xl |     20 | ts-guest-destroy             | 
pass
 leak-check/check          | test-amd64-i386-xl |     23 | ts-leak-check        
        | pass   | test-amd64-i386-xl |     21 | ts-leak-check                | 
pass
 capture-logs(24)          | test-amd64-i386-xl |     24 | ts-logs-capture      
        | pass   |                    |        |                              |
(25 rows)

osstestdb=>

Signed-off-by: Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>
---
 sg-run-job |    8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/sg-run-job b/sg-run-job
index 4f6b43a..a74d638 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -374,10 +374,7 @@ proc test-guest-migr {g} {
        run-ts . =.2 ts-guest-saverestore + host $g
     }
     if {$can_migrate} {
-        run-ts . =.2 ts-guest-localmigrate + host $g
-    }
-    if {$can_migrate} {
-        run-ts . = ts-guest-localmigrate x10 + host $g
+        run-ts . =/x10 ts-guest-localmigrate + x11 host $g
     }
 }
 
@@ -388,9 +385,8 @@ proc test-guest {g} {
 
 proc test-guest-nomigr {g} {
     run-ts . =   ts-guest-stop    + host $g
-    run-ts . =.2 ts-guest-start   + host $g
 
-    repeat-ts 10 =.repeat \
+    repeat-ts 11 =.repeat \
                + ts-guest-stop    host   $g \; \
                + ts-guest-start + host + $g +
 
-- 
1.7.10.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.