[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [PATCH v3 7/8] osstest: introduce a script to create a FreeBSD flight



This requires changes in several places in order to accommodate the
FreeBSD build, which although it's a build job, it doesn't have the
same set of dependencies as the current builds.

First, a new build recipe is added to sg-run-job, and accordingly
sg-run-job is also made aware about the differences between a Linux
and a FreeBSD build job. A Linux build job requires ts-host-allocate +
ts-host-install-twice + ts-xen-build-prep, while a FreeBSD build job
requires ts-freebsd-set-hostflags + ts-host-allocate +
ts-freebsd-host-install.

All the current build jobs are kept to use the Linux build recipe,
while the newly added build-freebsd is using the new FreeBSD recipe.

Finally, the logic to create a FreeBSD build job is added to
make-freebsd-flight. This includes creating a FreeBSD build job, and
also testing the output of that build job (by creating another build
job that depends on the output of the first).

Note that the FreeBSD build job needs some input in order to setup a
FreeBSD host, and that can be fetched from different places:

1. Env variable FREEBSD_BUILDJOB: use the output from a previous
build-<arch>-freebsd.

2. Env variables FREEBSD_DIST and FREEBSD_VERSION: set before calling
into make-flight, provide the path to the installer image and sets
folder and the version being installed.

3. Config file FreeBSDDist and FreeBSDVersion: same as 2. except that
they are set on the config file.

Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
---
Changes since v1:
 - Replace freebsd_buildjob with freebsdbuildjob.
 - Replace FREEBSD_SETS/IMAGE with a single FREEBSD_DIST that points
   to a folder that should contain both things.
 - Document the FreeBSDDist and FreeBSDVersion config file options in
   the README file.
---
 README              | 11 +++++++
 ap-common           |  4 +++
 make-freebsd-flight | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 sg-run-job          | 43 ++++++++++++++++++-------
 4 files changed, 138 insertions(+), 12 deletions(-)
 create mode 100755 make-freebsd-flight

diff --git a/README b/README
index b45058da..51243974 100644
--- a/README
+++ b/README
@@ -423,6 +423,17 @@ HostGroupFlags_<group>
 DebianPreseed
    Text to add to the debian-installer preseed file.  Optional.
 
+FreeBSDDist
+   Path to the folder that contains the FreeBSD install image and
+   the FreeBSD compressed install sets, together with the MANIFEST
+   file that holds the checksums. This is required in order to run
+   a FreeBSD host install if no previous FreeBSD buildjob is
+   available (ie: for example when running in standalone mode).
+
+FreeBSDVersion
+   Numeric value holding the major FreeBSD version of the media
+   provided in FreeBSDDist (ie: 12).
+
 ========================================
 
 Config settings relevant only to standalone mode
diff --git a/ap-common b/ap-common
index cbb815ce..d4fa7aef 100644
--- a/ap-common
+++ b/ap-common
@@ -37,6 +37,10 @@
 : ${PUSH_TREE_XTF:=$XENBITS:/home/xen/git/xtf.git}
 : ${BASE_TREE_XTF:=git://xenbits.xen.org/xtf.git}
 
+: ${TREE_FREEBSD:=git://github.com/freebsd/freebsd.git}
+: ${PUSH_TREE_FREEBSD:=$XENBITS:/home/xen/git/freebsd.git}
+: ${BASE_TREE_FREEBSD:=git://xenbits.xen.org/freebsd.git}
+
 : ${TREE_LIBVIRT:=git://libvirt.org/libvirt.git}
 : ${PUSH_TREE_LIBVIRT:=$XENBITS:/home/xen/git/libvirt.git}
 : ${BASE_TREE_LIBVIRT:=git://xenbits.xen.org/libvirt.git}
diff --git a/make-freebsd-flight b/make-freebsd-flight
new file mode 100755
index 00000000..93bb2cf3
--- /dev/null
+++ b/make-freebsd-flight
@@ -0,0 +1,92 @@
+#!/bin/bash
+
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2017 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+set -e -o posix
+
+branch=$1
+xenbranch=$2
+blessing=$3
+buildflight=$4
+
+flight=`./cs-flight-create $blessing $branch`
+
+. ./cri-common
+. ./ap-common
+. ./mfi-common
+
+arch=amd64
+
+job_create_build_filter_callback () {
+    :
+}
+
+get_freebsdjob_flags () {
+    arch=$1
+
+    # Figure out where are the installer binaries. The order is the following:
+    #
+    # 1. Env variable FREEBSD_BUILDJOB: use the output from a previous
+    # build-<arch>-freebsd.
+    #
+    # 2. Env variables FREEBSD_DIST, FREEBSD_VERSION: set before calling into
+    # make-flight, provide the path to the installer image, the sets to install
+    # and the version being installed.
+    #
+    # 3. Config file FreeBSDDist, FreeBSDVersion: same as 2. except that they
+    # are set on the config file.
+    #
+    if [ -n "$FREEBSD_BUILDJOB" ]; then
+        flags="freebsdbuildjob=$FREEBSD_BUILDJOB"
+    elif [ -n "$FREEBSD_DIST" ] && [ -n "$FREEBSD_VERSION" ]; then
+        flags="freebsd_distpath=$FREEBSD_DIST freebsd_version=$FREEBSD_VERSION"
+    else
+        distpath=`getconfig "FreeBSDDist"`
+        version=`getconfig "FreeBSDVersion"`
+        flags="freebsd_distpath=$distpath freebsd_version=$version"
+    fi
+
+    echo $flags
+}
+
+flags=`get_freebsdjob_flags $arch`
+job_create_build build-$arch-freebsd build-freebsd                       \
+            arch=$arch                                                   \
+            $RUNVARS $BUILD_RUNVARS $BUILD_FREEBSD_RUNVARS $arch_runvars \
+            tree_freebsd=$TREE_FREEBSD                                   \
+            revision_freebsd=$REVISION_FREEBSD                           \
+            extra_hostflags=arch-$arch,purpose-build                     \
+            $flags
+
+# Create an identical job that's going to use the build output from
+# the previous one.
+job_create_build build-$arch-freebsd-again build-freebsd                 \
+            arch=$arch                                                   \
+            $RUNVARS $BUILD_RUNVARS $BUILD_FREEBSD_RUNVARS $arch_runvars \
+            extra_hostflags=arch-$arch,purpose-build                     \
+            tree_freebsd=$TREE_FREEBSD                                   \
+            revision_freebsd=$REVISION_FREEBSD                           \
+            freebsdbuildjob=build-$arch-freebsd
+
+echo $flight
+
+# Local variables:
+# mode: sh
+# sh-basic-offset: 2
+# indent-tabs-mode: nil
+# End:
diff --git a/sg-run-job b/sg-run-job
index ceb79800..239446b8 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -53,12 +53,15 @@ proc run-job {job} {
     set skip_globs     [jobdb::read-runvar $flight $job skip_testids]
 
     set nh [need-hosts/$jobinfo(recipe)]
-    if {![string compare $nh BUILD]} {
+    if {![string compare $nh BUILD_LINUX]} {
         set need_xen_hosts {}
-        set need_build_host 1
+        set need_build_host LINUX
+    } elseif {![string compare $nh BUILD_FREEBSD]} {
+        set need_xen_hosts {}
+        set need_build_host FREEBSD
     } else {
         set need_xen_hosts $nh
-        set need_build_host 0
+        set need_build_host ""
     }
     set nested_layers_hosts {}
 
@@ -69,7 +72,11 @@ proc run-job {job} {
         eval run-ts broken  =             ts-hosts-allocate + $need_xen_hosts
     }
 
-    if {$need_build_host} { catching-otherwise broken prepare-build-host }
+    if {![string compare $need_build_host LINUX]} {
+        catching-otherwise broken prepare-build-host-linux
+    } elseif {![string compare $need_build_host FREEBSD]}  {
+        catching-otherwise broken prepare-build-host-freebsd
+    }
 
     if {$ok} { setstatus running                                          }
 
@@ -89,7 +96,7 @@ proc run-job {job} {
         set need_xen_hosts [lunappend nested_layers_hosts]
     }
 
-    if {$need_build_host && $anyfailed} {
+    if {[string length $need_build_host] && $anyfailed} {
        run-ts  !broken capture-logs      ts-logs-capture + host
     }
 
@@ -106,7 +113,7 @@ proc run-job {job} {
 
     if {$ok} { setstatus pass                                             }
 
-    if {$need_build_host && $ok} { jobdb::preserve-task 90 }
+    if {[string length $need_build_host] && $ok} { jobdb::preserve-task 90 }
 
     if {$anyfailed} {
         jobdb::logputs stdout "at least one test failed"
@@ -535,11 +542,12 @@ proc need-hosts/host-examine-linux {} {
 
 #---------- builds ----------
 
-proc need-hosts/build {} { return BUILD }
-proc need-hosts/build-kern {} { return BUILD }
-proc need-hosts/build-libvirt {} { return BUILD }
-proc need-hosts/build-rumprun {} { return BUILD }
-proc need-hosts/build-xtf {} { return BUILD }
+proc need-hosts/build {} { return BUILD_LINUX }
+proc need-hosts/build-kern {} { return BUILD_LINUX }
+proc need-hosts/build-libvirt {} { return BUILD_LINUX }
+proc need-hosts/build-rumprun {} { return BUILD_LINUX }
+proc need-hosts/build-xtf {} { return BUILD_LINUX }
+proc need-hosts/build-freebsd {} { return BUILD_FREEBSD }
 
 proc run-job/build {} {
     run-ts . = ts-xen-build
@@ -566,13 +574,24 @@ proc run-job/build-xtf {} {
     run-ts . = ts-xtf-build
 }
 
-proc prepare-build-host {} {
+proc run-job/build-freebsd {} {
+    run-ts . = ts-freebsd-build
+}
+
+proc prepare-build-host-linux {} {
     global jobinfo
     run-ts broken = ts-hosts-allocate + host
     run-ts broken host-install(*) ts-host-install-twice
     run-ts . host-build-prep ts-xen-build-prep
 }
 
+proc prepare-build-host-freebsd {} {
+    global jobinfo
+    run-ts broken = ts-freebsd-set-hostflags
+    run-ts broken = ts-hosts-allocate + host
+    run-ts broken host-install(*) ts-freebsd-host-install
+}
+
 proc need-hosts/coverity {} { return BUILD }
 proc run-job/coverity {} {
     run-ts . = ts-coverity-build + host
-- 
2.11.0 (Apple Git-81)


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.