[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [PATCH OSSTEST v2] Arrange to test migration from the previous Xen version



There are several steps to this:

- Identify $prevxenbranch, that is the branch which precedes
  $xenbranch.
- Create appropriate build jobs.
- Add support in ts-xen-install for overriding {xen,}buildjob on a
  per-ident basis
- Add a new receipt test-pair-oneway which only migrates from
  src_host to dst_host and not the reverse
- Create appropriate test jobs, overridding the default builds for
  src_host.

Currently we only do this for xen* branches and using xl, but in the
future we may wish to add to the libvirt branch too.

It would be nice to try and reuse the builds from the last flight
which tested the $prevxenbranch baseline. I've not dont that here.

+build-amd64-prev                                      arch                     
   amd64
+build-amd64-prev                                      build_lvextend_max       
   50
+build-amd64-prev                                      enable_ovmf              
   true
+build-amd64-prev                                      enable_xend              
   false
+build-amd64-prev                                      enable_xsm               
   false
+build-amd64-prev                                      host_hostflags           
   share-build-wheezy-amd64,arch-amd64,suite-wheezy,purpose-build
+build-amd64-prev                                      revision_ovmf
+build-amd64-prev                                      revision_qemu
+build-amd64-prev                                      revision_qemuu           
   c4a962ec0c61aa9b860a3635c8424472e6c2cc2c
+build-amd64-prev                                      revision_seabios
+build-amd64-prev                                      revision_xen             
   666b80f239c566283cb1b3435180d99a329d0156
+build-amd64-prev                                      tree_ovmf
+build-amd64-prev                                      tree_qemu                
   git://xenbits.xen.org/staging/qemu-xen-unstable.git
+build-amd64-prev                                      tree_qemuu               
   git://xenbits.xen.org/staging/qemu-upstream-unstable.git
+build-amd64-prev                                      tree_seabios
+build-amd64-prev                                      tree_xen                 
   git://xenbits.xen.org/xen.git

+build-i386-prev                                       arch                     
   i386
+build-i386-prev                                       build_lvextend_max       
   50
+build-i386-prev                                       enable_ovmf              
   true
+build-i386-prev                                       enable_xend              
   false
+build-i386-prev                                       enable_xsm               
   false
+build-i386-prev                                       host_hostflags           
   share-build-wheezy-i386,arch-i386,suite-wheezy,purpose-build
+build-i386-prev                                       revision_ovmf
+build-i386-prev                                       revision_qemu
+build-i386-prev                                       revision_qemuu           
   c4a962ec0c61aa9b860a3635c8424472e6c2cc2c
+build-i386-prev                                       revision_seabios
+build-i386-prev                                       revision_xen             
   666b80f239c566283cb1b3435180d99a329d0156
+build-i386-prev                                       tree_ovmf
+build-i386-prev                                       tree_qemu                
   git://xenbits.xen.org/staging/qemu-xen-unstable.git
+build-i386-prev                                       tree_qemuu               
   git://xenbits.xen.org/staging/qemu-upstream-unstable.git
+build-i386-prev                                       tree_seabios
+build-i386-prev                                       tree_xen                 
   git://xenbits.xen.org/xen.git

+test-amd64-amd64-upgrade                              all_hostflags            
   arch-amd64,arch-xen-amd64,suite-wheezy,purpose-test,equiv-1
+test-amd64-amd64-upgrade                              arch                     
   amd64
+test-amd64-amd64-upgrade                              buildjob                 
   build-amd64
+test-amd64-amd64-upgrade                              debian_arch              
   amd64
+test-amd64-amd64-upgrade                              debian_kernkind          
   pvops
+test-amd64-amd64-upgrade                              kernbuildjob             
   build-amd64-pvops
+test-amd64-amd64-upgrade                              kernkind                 
   pvops
+test-amd64-amd64-upgrade                              src_host_buildjob        
   build-amd64-prev
+test-amd64-amd64-upgrade                              src_host_xenbuildjob     
   build-amd64-prev
+test-amd64-amd64-upgrade                              toolstack                
   xl
+test-amd64-amd64-upgrade                              xenbuildjob              
   build-amd64

+test-amd64-i386-upgrade                               all_hostflags            
   arch-i386,arch-xen-amd64,suite-wheezy,purpose-test,equiv-1
+test-amd64-i386-upgrade                               arch                     
   i386
+test-amd64-i386-upgrade                               buildjob                 
   build-i386
+test-amd64-i386-upgrade                               debian_arch              
   i386
+test-amd64-i386-upgrade                               debian_kernkind          
   pvops
+test-amd64-i386-upgrade                               kernbuildjob             
   build-i386-pvops
+test-amd64-i386-upgrade                               kernkind                 
   pvops
+test-amd64-i386-upgrade                               src_host_buildjob        
   build-i386-prev
+test-amd64-i386-upgrade                               src_host_xenbuildjob     
   build-amd64-prev
+test-amd64-i386-upgrade                               toolstack                
   xl
+test-amd64-i386-upgrade                               xenbuildjob              
   build-amd64

NB the regular build jobs are, as expected, unchanged (and different to the
ones above):

 build-amd64                                           revision_xen             
   9256f66c1606cd9339412bff7fbc7bd9e8beb28c
 build-i386                                            revision_xen             
   9256f66c1606cd9339412bff7fbc7bd9e8beb28c

Signed-off-by: Ian Campbell <ian.campbell@xxxxxxxxxx>
---
v2:
 - Don't do this for Xen 4.0 and earlier
 - Update some comments and change some XXX to TODO
 - Match branch against xen-*
---
 cr-daily-branch |  4 ++++
 cri-common      | 20 ++++++++++++++++++++
 make-flight     | 16 ++++++++++++++++
 mfi-common      | 38 ++++++++++++++++++++++++++++++++++++++
 sg-run-job      | 14 ++++++++++++--
 ts-xen-install  |  3 ++-
 6 files changed, 92 insertions(+), 3 deletions(-)

diff --git a/cr-daily-branch b/cr-daily-branch
index 1fcfd9d..7402d3f 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -117,6 +117,10 @@ if [ "x$REVISION_XEN" = x ]; then
         REVISION_XEN="`fetch_version $xenbranch`"
         export REVISION_XEN
 fi
+if [ "x$REVISION_PREVXEN" = x -a "x$prevxenbranch" != "x" ]; then
+       REVISION_PREVXEN="`./ap-fetch-version-baseline $prevxenbranch`"
+       export REVISION_PREVXEN
+fi
 if [ "x$REVISION_QEMU_UPSTREAM" = x ]; then
        case "$xenbranch" in
         xen-3.*|xen-4.[01]-*)
diff --git a/cri-common b/cri-common
index 5c58482..94696ab 100644
--- a/cri-common
+++ b/cri-common
@@ -60,6 +60,24 @@ repo_tree_rev_fetch_git () {
        git rev-parse $localtag^0
 }
 
+select_prevxenbranch () {
+       local b
+       local p
+       for b in $(./mg-list-all-branches) ; do # already sorted by version
+               case "$b" in
+               xen*)
+                       if [ "x$b" = "x$xenbranch" ] ; then
+                               break
+                       else
+                               p=$b
+                       fi
+                       ;;
+               *)      ;;
+               esac
+       done
+       prevxenbranch=$p
+}
+
 select_xenbranch () {
        case "$branch" in
        xen-*)                  tree=xen;       xenbranch=$branch ;;
@@ -83,6 +101,8 @@ select_xenbranch () {
        if [ x$qemuubranch = x ]; then
                qemuubranch="qemu-upstream-${xenbranch#xen-}"
        fi
+
+       select_prevxenbranch
 }
 
 select_branch () {
diff --git a/make-flight b/make-flight
index edca13f..7bf5bc8 100755
--- a/make-flight
+++ b/make-flight
@@ -520,6 +520,22 @@ test_matrix_do_one () {
             $debian_runvars \
             all_hostflags=$most_hostflags,equiv-1
 
+  if branch_wants_migrupgrade_tests ; then
+      # x86 only for now
+      if [ x$xenarch = xamd64 ] ; then
+      job_create_test test-$xenarch$kern-$dom0arch-migrupgrade 
test-pair-oneway \
+                      $onetoolstack $xenarch $dom0arch \
+                      !host !host_hostflags \
+                      src_host_buildjob=${prevbfi}build-$dom0arch-prev \
+                      src_host_xenbuildjob=${prevbfi}build-$xenarch-prev \
+                      $debian_runvars \
+                      all_hostflags=$most_hostflags,equiv-1
+      fi
+      # TODO it would be nice to test i386->amd64 tools migration too (for
+      # Xen >= 4.6), but currently $r{arch} is used in several places,
+      # which would need looking at.
+  fi
+
   if [ x$test_pvh = xy -a $xenarch = amd64 -a $dom0arch = amd64 ]; then
 
     for cpuvendor in amd intel; do
diff --git a/mfi-common b/mfi-common
index 23f8015..28a39fd 100644
--- a/mfi-common
+++ b/mfi-common
@@ -41,6 +41,15 @@ branch_wants_rumpkernel_tests () {
   esac
 }
 
+branch_wants_migrupgrade_tests () {
+  case "$branch" in
+    xen-3.*-testing) return 1 ;;
+    xen-4.0-testing) return 1 ;;
+    xen-*) return 0;;
+    *) return 1 ;;
+  esac
+}
+
 xenbranch_xsm_variants () {
     case "$xenbranch" in
     xen-3.*-testing) echo "false";;
@@ -161,6 +170,14 @@ create_build_jobs () {
     *) enable_ovmf=true;
     esac
 
+    want_prevxen=n
+    if branch_wants_migrupgrade_tests ; then
+        # Only x86 for now
+        if [ x$arch = xamd64 -o x$arch = xi386 ] ; then
+            want_prevxen=y
+        fi
+    fi
+
     eval "
         arch_runvars=\"\$ARCH_RUNVARS_$arch\"
     "
@@ -191,6 +208,27 @@ create_build_jobs () {
                 revision_ovmf=$REVISION_OVMF
     done
 
+    if [ x$want_prevxen = xy ] ; then
+        if [ "x$REVISION_PREVXEN" = x ] ; then
+            echo >&2 "prevxen ?"; exit 1
+        fi
+        # TODO could find latest pass on that branch and attempt to reuse.
+        #bfiprevxen=...
+        #
+        # To avoid the need for TREE_PREVQEMU, TREE_PREVOVMF etc we
+        # only pass tree_xen and revision_xen here and therefore
+        # pickup the versions implicitly selected by the
+        # $REVISION_PREVXEN.
+        ./cs-job-create $flight build-$arch-prev build                       \
+                    arch=$arch enable_xend=false enable_ovmf=$enable_ovmf    \
+                    enable_xsm=false                                         \
+            tree_xen=$TREE_XEN                                               \
+                    $RUNVARS $BUILD_RUNVARS $BUILD_XEN_RUNVARS $arch_runvars \
+                    $suite_runvars                                           \
+                    host_hostflags=$build_hostflags                          \
+                    revision_xen=$REVISION_PREVXEN
+    fi
+
     if [ $build_extraxend = "true" ] ; then
     job_create_build build-$arch-xend build                                  \
                 arch=$arch enable_xend=true enable_ovmf=$enable_ovmf         \
diff --git a/sg-run-job b/sg-run-job
index bba52ed..385e937 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -294,13 +294,16 @@ proc run-job/test-debianhvm {} {
     test-guest debianhvm
 }
 
-proc need-hosts/test-pair {} { return {src_host dst_host} }
-proc run-job/test-pair {} {
+proc setup-test-pair {} {
     run-ts . =              ts-debian-install      dst_host
     run-ts . =              ts-debian-fixup        dst_host          + debian
     run-ts . =              ts-guests-nbd-mirror + dst_host src_host + debian
     per-host-ts . =(*)     {ts-leak-check basis}
     run-ts . =              ts-guest-start       + src_host          + debian
+}
+proc need-hosts/test-pair {} { return {src_host dst_host} }
+proc run-job/test-pair {} {
+    setup-test-pair
     run-ts . =              ts-guest-migrate       src_host dst_host + debian
     run-ts . =              ts-guest-migrate       dst_host src_host + debian
     run-ts . =              ts-guest-stop          src_host          + debian
@@ -310,6 +313,13 @@ proc run-job/test-pair {} {
 #    run-ts . remus-failover ts-remus-check         src_host dst_host + debian
 }
 
+proc need-hosts/test-pair-oneway {} { return {src_host dst_host} }
+proc run-job/test-pair-oneway {} {
+    setup-test-pair
+    run-ts . =              ts-guest-migrate       src_host dst_host + debian
+    run-ts . =              ts-guest-stop          dst_host          + debian
+}
+
 proc test-guest-migr {g} {
     if {[catch { run-ts . = ts-migrate-support-check + host $g }]} return
 
diff --git a/ts-xen-install b/ts-xen-install
index 0f53382..e9e4ada 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -75,7 +75,8 @@ sub extract () {
 
     foreach my $part (@parts) {
         target_extract_jobdistpath($ho, $part, "path_${part}dist",
-                                  $r{"${part}buildjob"}, \%distpath);
+               $r{"$ho->{Ident}_${part}buildjob"} // $r{"${part}buildjob"},
+               \%distpath);
     }
     if (target_file_exists($ho, "/usr/lib64/efi/xen.efi")) {
        target_cmd_root($ho,<<END);
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.