[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] [PATCH osstest.git v3] Add libvirt build and test jobs.



The ts-libvirt-build, sg-run-job and mfi-common changes have been tested in
standalone mode.

The ts-xen-install change has been tested in standalone mode, it is expected
that the rest of test-*-*-libvirt will fail horribly at this time.

The remaining changes are intended to introduce a libvirt push gate and have
been tested as far as my limited ability to invoke cr-daily-branch by hand is
able, but I am sure they are incomplete in some way or other.

The libvirt.git initscript is horribly redhat specific, and the debian package
one is large and complex, so ts-libvirt-build inserts its own simple
initscript.

The standalone wrapper is updated to handle path_libvirt in set-paths.

libvirt.git's autogen.sh involves initialising/cloning a git submodule. AIUI
the way git submodules work they are effectively versioned by the repo which
references them and so there are no push gate implications here.

Signed-off-by: Ian Campbell <ian.campbell@xxxxxxxxxx>
Acked-by: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
Cc: Jim Fehlig <jfehlig@xxxxxxxx>
Cc: Dario Faggioli <dario.faggioli@xxxxxxxxxx>
---
v3: Disable uml, lxc and vbox backends at build time
    Create test job on armhf as well as x86
    Don't run non-libvirt tests in libvirt flights.
v2: Add a test job, which will clearly fail.
    Make ts-xen-install DTRT though
---
 Osstest/TestSupport.pm |   7 ++-
 ap-common              |   5 ++
 ap-fetch-version       |   4 ++
 ap-fetch-version-old   |   5 ++
 ap-print-url           |   3 +
 ap-push                |   5 ++
 cr-daily-branch        |   8 +++
 cri-common             |   1 +
 make-flight            |   7 +++
 mfi-common             |  15 ++++-
 sg-run-job             |   5 ++
 standalone             |   2 +-
 ts-libvirt-build       | 159 +++++++++++++++++++++++++++++++++++++++++++++++++
 ts-xen-install         |   5 +-
 14 files changed, 227 insertions(+), 4 deletions(-)
 create mode 100755 ts-libvirt-build

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index b11eb17..87b8b2e 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1764,7 +1764,12 @@ our %toolstacks=
         Command => 'xl',
         CfgPathVar => 'cfgpath',
        RestoreNeedsConfig => 1,
-        }
+        },
+     'libvirt' => {
+        NewDaemons => [qw(libvirtd)],
+        Dom0MemFixed => 1,
+        Command => 'virsh',
+        },
      );
 
 sub toolstack () {
diff --git a/ap-common b/ap-common
index cb0cfff..73bb90c 100644
--- a/ap-common
+++ b/ap-common
@@ -33,6 +33,10 @@
 
 : ${TREEVCS_LINUX:=git}
 
+: ${TREE_LIBVIRT:=git://libvirt.org/libvirt.git}
+: ${PUSH_TREE_LIBVIRT:=$XENBITS:/home/xen/git/libvirt.git}
+: ${TREEVCS_LIBVIRT:=git}
+
 : ${TREE_LINUXFIRMWARE:=git://xenbits.xen.org/osstest/linux-firmware.git}
 : ${PUSH_TREE_LINUXFIRMWARE:=$XENBITS:/home/osstest/ext/linux-firmware.git}
 : 
${UPSTREAM_TREE_LINUXFIRMWARE:=$GIT_KERNEL_ORG/pub/scm/linux/kernel/git/firmware/linux-firmware.git}
@@ -56,6 +60,7 @@ fi
 
 : ${LOCALREV_XEN:=daily-cron.$branch}
 : ${LOCALREV_LINUX:=daily-cron.$branch}
+: ${LOCALREV_LIBVIRT:=daily-cron.$branch}
 
 : ${TREEBASE_LINUX_XCP:=http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27}
 
diff --git a/ap-fetch-version b/ap-fetch-version
index aaf884d..2990462 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -69,6 +69,10 @@ linuxfirmware)
        repo_tree_rev_fetch_git linuxfirmware \
                $UPSTREAM_TREE_LINUXFIRMWARE master daily-cron.$branch
        ;;
+libvirt)
+       repo_tree_rev_fetch_git libvirt \
+               $TREE_LIBVIRT master $LOCALREV_LIBVIRT
+       ;;
 osstest)
        if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
            git fetch $HOME/testing.git pretest:ap-fetch >&2
diff --git a/ap-fetch-version-old b/ap-fetch-version-old
index 21ff786..a32de0e 100755
--- a/ap-fetch-version-old
+++ b/ap-fetch-version-old
@@ -27,6 +27,7 @@ select_xenbranch
 
 : ${BASE_TAG_LINUX2639:=tested/2.6.39.x}
 : ${BASE_LOCALREV_LINUX:=daily-cron.$branch.old}
+: ${BASE_LOCALREV_LIBVIRT:=daily-cron.$branch.old}
 
 : ${BASE_TREE_QEMU_UPSTREAM:=${TREE_QEMU_UPSTREAM/\/staging\//\/}}
 
@@ -73,6 +74,10 @@ linuxfirmware)
        repo_tree_rev_fetch_git linuxfirmware \
                $TREE_LINUXFIRMWARE master daily-cron-old.$branch
        ;;
+libvirt)
+       repo_tree_rev_fetch_git libvirt \
+               $TREE_LIBVIRT master $BASE_LOCALREV_LIBVIRT
+       ;;
 osstest)
        if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
            git fetch -f $HOME/testing.git incoming:ap-fetch
diff --git a/ap-print-url b/ap-print-url
index f2812d0..2b55fce 100755
--- a/ap-print-url
+++ b/ap-print-url
@@ -46,6 +46,9 @@ linux-2.6.39)
 linuxfirmware)
        echo $UPSTREAM_TREE_LINUXFIRMWARE
        ;;
+libvirt)
+       echo $TREE_LIBVIRT
+       ;;
 osstest)
        echo none:;
        ;;
diff --git a/ap-push b/ap-push
index dc0a166..af76ecc 100755
--- a/ap-push
+++ b/ap-push
@@ -32,6 +32,7 @@ select_xenbranch
 TREE_LINUX=$PUSH_TREE_LINUX
 TREE_QEMU_UPSTREAM=$XENBITS:/home/xen/git/qemu-upstream-${xenbranch#xen-}.git
 TREE_XEN=$PUSH_TREE_XEN
+TREE_LIBVIRT=$PUSH_TREE_LIBVIRT
 
 if info_linux_tree "$branch"; then
        cd $repos/linux
@@ -72,6 +73,10 @@ linux-xen-*)
         cd $repos/linux
         git push $TREE_LINUX $revision:refs/heads/$branch
         ;;
+libvirt)
+       cd $repos/libvirt
+       git push $TREE_LIBVIRT $revision:master
+       ;;
 osstest)
        git push $HOME/testing.git $revision:incoming
        git push $XENBITS:/home/xen/git/osstest.git $revision:master
diff --git a/cr-daily-branch b/cr-daily-branch
index 02fef15..a13d75a 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -133,6 +133,10 @@ if [ "x$REVISION_LINUX_ARM" = x ]; then
         export REVISION_LINUX_ARM
     fi
 fi
+if [ "x$REVISION_LIBVIRT" = x ]; then
+       determine_version REVISION_LIBVIRT libvirt
+       export REVISION_LIBVIRT
+fi
 if [ "x$REVISION_LINUXFIRMWARE" = x ]; then
        determine_version REVISION_LINUXFIRMWARE linuxfirmware
         export REVISION_LINUXFIRMWARE
@@ -165,6 +169,10 @@ linuxfirmware)
        realtree=linuxfirmware
        NEW_REVISION=$REVISION_LINUXFIRMWARE
         ;;
+libvirt)
+       realtree=libvirt
+       NEW_REVISION=$REVISION_LIBVIRT
+       ;;
 *)
        NEW_REVISION=''
        wantpush=false
diff --git a/cri-common b/cri-common
index 497d4e3..f472927 100644
--- a/cri-common
+++ b/cri-common
@@ -42,6 +42,7 @@ select_xenbranch () {
        linux)                  tree=linux;     xenbranch=xen-unstable ;;
        linux-*)                tree=linux;     xenbranch=xen-unstable ;;
        linuxfirmware)      tree=linuxfirmware; xenbranch=xen-unstable ;;
+       libvirt)                tree=libvirt;   xenbranch=xen-unstable ;;
        osstest)                tree=osstest;   xenbranch=xen-unstable ;;
        esac
        if [ "x$tree" = xlinux ]; then
diff --git a/make-flight b/make-flight
index 92b813a..aadf58b 100755
--- a/make-flight
+++ b/make-flight
@@ -64,6 +64,9 @@ job_create_test_filter_callback () {
           return 1;;
       esac
       ;;
+    libvirt)
+      if [ x$toolstack != xlibvirt ] ; then return 1; fi
+      ;;
     *)
       case "$job" in
         *-qemuu-*)
@@ -238,6 +241,10 @@ test_matrix_do_one () {
             $xenarch $dom0arch                                   \
             $debian_runvars all_hostflags=$most_hostflags
 
+  job_create_test test-$xenarch$kern-$dom0arch-libvirt test-debian libvirt \
+            $xenarch $dom0arch                                       \
+            $debian_runvars all_hostflags=$most_hostflags
+
   # No further arm tests at the moment
   if [ $dom0arch = armhf ]; then
       return
diff --git a/mfi-common b/mfi-common
index 598aa02..a8e528a 100644
--- a/mfi-common
+++ b/mfi-common
@@ -162,6 +162,16 @@ create_build_jobs () {
                 tree_linuxfirmware=$TREE_LINUXFIRMWARE                       \
                 revision_linuxfirmware=$REVISION_LINUXFIRMWARE
 
+    ./cs-job-create $flight build-$arch-libvirt build-libvirt                \
+                arch=$arch                                                   \
+        tree_xen=$TREE_XEN                                                   \
+                $RUNVARS $BUILD_RUNVARS $BUILD_LIBVIRT_RUNVARS $arch_runvars \
+                $suite_runvars                                               \
+                host_hostflags=$build_hostflags                              \
+                buildjob=${bfi}build-$arch                                   \
+                tree_libvirt=$TREE_LIBVIRT revision_libvirt=$REVISION_LIBVIRT\
+                ${TREEVCS_LIBVIRT:+treevcs_libvirt=}${TREEVCS_LIBVIRT}
+
     case "$arch" in
     armhf) continue;; # don't do any other kernel builds
     esac
@@ -193,6 +203,7 @@ job_create_test () {
 
   xenbuildjob="${bfi}build-$xenarch"
   buildjob="${bfi}build-$dom0arch"
+  tsbuildjob=
 
   case "$xenbranch:$toolstack" in
     xen-3.*-testing:*) ;;
@@ -203,11 +214,13 @@ job_create_test () {
     *:xend) xenbuildjob="$xenbuildjob-xend"
             buildjob="${bfi}build-$dom0arch-xend"
             ;;
+    *:libvirt) tsbuildjob="libvirtbuildjob=$buildjob-libvirt"
+            ;;
   esac
 
   ./cs-job-create $flight $job $recipe toolstack=$toolstack       \
     $RUNVARS $TEST_RUNVARS $most_runvars                          \
-    xenbuildjob=$xenbuildjob buildjob=$buildjob "$@"
+    xenbuildjob=$xenbuildjob buildjob=$buildjob $tsbuildjob "$@"
 }
 
 # Iterate over xenarch, dom0arch and kernel calling test_matrix_do_one
diff --git a/sg-run-job b/sg-run-job
index d336fbd..3d44331 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -312,6 +312,7 @@ proc test-guest-nomigr {g} {
 
 proc need-hosts/build {} { return BUILD }
 proc need-hosts/build-kern {} { return BUILD }
+proc need-hosts/build-libvirt {} { return BUILD }
 
 proc run-job/build {} {
     run-ts . = ts-xen-build
@@ -321,6 +322,10 @@ proc run-job/build-kern {} {
     run-ts . = ts-kernel-build
 }
 
+proc run-job/build-libvirt {} {
+    run-ts . = ts-libvirt-build
+}
+
 proc prepare-build-host {} {
     global jobinfo
     run-ts broken = ts-hosts-allocate + host
diff --git a/standalone b/standalone
index 37c0294..d10a223 100755
--- a/standalone
+++ b/standalone
@@ -166,7 +166,7 @@ case $op in
 
        job=$1; shift
 
-       for d in '' xen kern ; do
+       for d in '' xen kern libvirt ; do
            runvar="path_${d}dist"
            path="build/${d}dist.tar.gz"
            if [ -f "logs/$flight/$job/$path" ] ; then
diff --git a/ts-libvirt-build b/ts-libvirt-build
new file mode 100755
index 0000000..56b78f1
--- /dev/null
+++ b/ts-libvirt-build
@@ -0,0 +1,159 @@
+#!/usr/bin/perl
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2014 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho= selecthost($whhost);
+
+my $leaf= "build.$flight.$job";
+my $homedir = get_host_property($ho, 'homedir', '/home/osstest');
+my $builddir= "$homedir/$leaf";
+my $xendist= "$builddir/xendist";
+
+sub libvirtd_init ();
+
+my $makeflags= get_host_property($ho, 'build make flags', '-j4');
+
+sub checkout () {
+    target_cmd($ho, "rm -rf $builddir && mkdir $builddir && mkdir $xendist", 
600);
+
+    my $path = get_stashed("path_dist", $r{"buildjob"});
+    my $distcopy= "$builddir/dist.tar.gz";
+    target_putfile_root($ho, 300, $path, $distcopy);
+    target_cmd_root($ho, "tar -C $xendist -hzxf $distcopy", 300);
+
+    build_clone($ho, 'libvirt', $builddir, 'libvirt');
+}
+
+sub prep() {
+    target_install_packages($ho, qw(autoconf automake libtool xsltproc
+                                    libxml2-utils libxml2-dev libnl-dev
+                                    libdevmapper-dev w3c-dtd-xhtml));
+}
+
+sub config() {
+    target_cmd_build($ho, 1000, $builddir, <<END);
+        cd libvirt
+        CFLAGS="-I$xendist/usr/local/include/" \\
+        LDFLAGS="-L$xendist/usr/local/lib/ 
-Wl,-rpath-link=$xendist/usr/local/lib/" \\
+            ./autogen.sh --with-libxl --without-xen --without-xenapi 
--without-selinux \\
+                         --without-lxc --without-vbox --without-uml \\
+                         --sysconfdir=/etc --localstatedir=/var #/
+END
+}
+
+sub build() {
+    target_cmd_build($ho, 1000, $builddir, <<END);
+        cd libvirt
+        (make $makeflags 2>&1 && touch ../build-ok-stamp) |tee ../log
+        test -f ../build-ok-stamp #/
+        echo ok.
+END
+}
+
+sub install() {
+    target_cmd_build($ho, 300, $builddir, <<END);
+        mkdir -p dist
+        cd libvirt
+        make $makeflags install DESTDIR=$builddir/dist
+        mkdir -p $builddir/dist/etc/init.d
+END
+    target_putfilecontents_stash($ho, 60,
+                                 libvirtd_init(),
+                                 $builddir.'/dist/etc/init.d/libvirtd');
+    target_cmd_build($ho, 60, $builddir, <<END);
+        chmod +x $builddir/dist/etc/init.d/libvirtd
+END
+}
+
+prep();
+checkout();
+config();
+build();
+install();
+built_stash($ho, $builddir, 'dist', 'libvirtdist');
+
+sub libvirtd_init () {
+    return <<'END';
+#! /bin/sh
+#
+# Init script for libvirtd
+#
+# Loosely based on Debian libvirt-bin initscript:
+# (c) 2007 Guido Guenther <agx@xxxxxxxxxxx>
+# based on the skeletons that comes with dh_make
+#
+### BEGIN INIT INFO
+# Provides:          libvirt-bin libvirtd
+# Required-Start:    $network $local_fs $remote_fs $syslog
+# Required-Stop:     $local_fs $remote_fs $syslog
+# Should-Start:      avahi-daemon cgconfig
+# Should-Stop:       avahi-daemon cgconfig
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: libvirt management daemon
+### END INIT INFO
+
+PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+export PATH
+DAEMON=/usr/local/sbin/libvirtd
+NAME=libvirtd
+DESC="libvirt management daemon"
+
+test -x $DAEMON || exit 0
+. /lib/lsb/init-functions
+
+PIDFILE=/var/run/$NAME.pid
+
+case "$1" in
+  start)
+       [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
+       start-stop-daemon --start --pidfile $PIDFILE \
+           --exec $DAEMON -- -d $libvirtd_opts
+       [ "$VERBOSE" != no ] && log_end_msg $?
+       ;;
+  stop)
+       [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
+       start-stop-daemon --stop --pidfile $PIDFILE \
+           --exec $DAEMON
+       [ "$VERBOSE" != no ] && log_end_msg $?
+       ;;
+  restart)
+       stop
+       start
+       ;;
+  reload|force-reload)
+       start-stop-daemon --stop --signal 1 --quiet --pidfile \
+            /var/run/$NAME.pid --exec $DAEMON
+       ;;
+  *)
+       N=/etc/init.d/libvirtd
+       echo "Usage: $N {start|stop|restart|reload|force-reload}" >&2
+       exit 1
+       ;;
+esac
+
+exit 0
+END
+}
diff --git a/ts-xen-install b/ts-xen-install
index 4a810e4..435fc3e 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -63,7 +63,10 @@ sub packages () {
 }
 
 sub extract () {
-    foreach my $part ('', 'kern', 'xen') {
+    my @parts = ('', 'kern', 'xen');
+    push @parts, 'libvirt' if $r{toolstack} eq "libvirt";
+
+    foreach my $part (@parts) {
         target_extract_jobdistpath($ho, $part, "path_${part}dist",
                                   $r{"${part}buildjob"}, \%distpath);
     }
-- 
1.9.0


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.