|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-devel] [PATCH osstest v5] Add libvirt build and test jobs.
I have run this through a "cr-daily-branch --real" and the push gate machinery
seem functional. The test cases run through to starting a guest which fails
because virsh support has not been implemented yet.
The libvirt.git initscript is horribly redhat specific, and the debian package
one is large and complex, so ts-libvirt-build inserts its own simple
initscript.
I have observed the autogen.sh phase taking 34 minutes on army (arm build
cluster) hence the surprisingly large timeout of an hour. The build took around
20 minutes on the same system, but I used a timeout of an hour anyway to be
safe.
The standalone wrapper is updated to handle path_libvirt in set-paths.
libvirt.git's autogen.sh involves initialising/cloning a git submodule, which
doesn't use osstest's git cache and introduces unpredicability into what is
built. Therefore ap-* are taught about a new libvirt-gnulib tree (prefixed
libvirt incase we end up with other users of gnulib in the future). However
this doesn't use the usual push gate mechanism, since this was deemed to be
overkill and just always uses the tip of gnulib. The results are recorded
though so the bisector can still operate over this tree in the event of
breakage.
Signed-off-by: Ian Campbell <ian.campbell@xxxxxxxxxx>
Acked-by: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
Cc: Jim Fehlig <jfehlig@xxxxxxxx>
Cc: Dario Faggioli <dario.faggioli@xxxxxxxxxx>
---
v5: Print message in ap-push when not pushing gnulib
Switch from libvirt-gnulib to gnulib-libvirt in all tree names. Implemented
with:
sed -i -e 's/\(libvirt\)\([-_]\)\(gnulib\)/\3\2\1/gi' $(git grep -il
libvirt.gnulib)
And a manual inspection of the result.
v4: Use "xen-tested-master" as the push gate output.
Install extra libraries needed by the binary onto the target.
Install build dependencies in ts-xen-build-prep instead of
ts-libvirt-build. Although apt-get invocations are now locked I still have
concerns:
http://lists.xen.org/archives/html/xen-devel/2014-04/msg01951.html
Clone gnulib separately.
Avoid running stuff as root during build, no need
Correct ap-fetch-version-old (actually use the gate, not the main libvirt
tree)
Increase timeouts for arm build.
Ran through cr-daily-branch
v3: Disable uml, lxc and vbox backends at build time
Create test job on armhf as well as x86
Don't run non-libvirt tests in libvirt flights.
v2: Add a test job, which will clearly fail.
Make ts-xen-install DTRT though
---
Osstest/TestSupport.pm | 8 ++-
ap-common | 10 +++
ap-fetch-version | 8 +++
ap-fetch-version-old | 9 +++
ap-print-url | 6 ++
ap-push | 9 +++
cr-daily-branch | 12 ++++
cri-common | 1 +
make-flight | 7 +++
mfi-common | 17 +++++-
sg-run-job | 5 ++
standalone | 2 +-
ts-libvirt-build | 162 +++++++++++++++++++++++++++++++++++++++++++++++++
ts-xen-build-prep | 3 +
ts-xen-install | 7 ++-
15 files changed, 262 insertions(+), 4 deletions(-)
create mode 100755 ts-libvirt-build
diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index d49ff4f..4ea59a6 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -1774,7 +1774,13 @@ our %toolstacks=
Command => 'xl',
CfgPathVar => 'cfgpath',
RestoreNeedsConfig => 1,
- }
+ },
+ 'libvirt' => {
+ NewDaemons => [qw(libvirtd)],
+ Dom0MemFixed => 1,
+ Command => 'virsh',
+ ExtraPackages => [qw(libnl1 libavahi-client3)],
+ },
);
sub toolstack () {
diff --git a/ap-common b/ap-common
index cb0cfff..173f4be 100644
--- a/ap-common
+++ b/ap-common
@@ -33,6 +33,14 @@
: ${TREEVCS_LINUX:=git}
+: ${TREE_LIBVIRT:=git://libvirt.org/libvirt.git}
+: ${PUSH_TREE_LIBVIRT:=$XENBITS:/home/xen/git/libvirt.git}
+: ${BASE_TREE_LIBVIRT:=git://xenbits.xen.org/libvirt.git}
+: ${TREEVCS_LIBVIRT:=git}
+
+: ${TREE_GNULIB_LIBVIRT:=git://git.sv.gnu.org/gnulib.git}
+: ${TREEVCS_GNULIB_LIBVIRT:=git}
+
: ${TREE_LINUXFIRMWARE:=git://xenbits.xen.org/osstest/linux-firmware.git}
: ${PUSH_TREE_LINUXFIRMWARE:=$XENBITS:/home/osstest/ext/linux-firmware.git}
:
${UPSTREAM_TREE_LINUXFIRMWARE:=$GIT_KERNEL_ORG/pub/scm/linux/kernel/git/firmware/linux-firmware.git}
@@ -56,6 +64,8 @@ fi
: ${LOCALREV_XEN:=daily-cron.$branch}
: ${LOCALREV_LINUX:=daily-cron.$branch}
+: ${LOCALREV_LIBVIRT:=daily-cron.$branch}
+: ${LOCALREV_GNULIB_LIBVIRT:=daily-cron.$branch}
: ${TREEBASE_LINUX_XCP:=http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27}
diff --git a/ap-fetch-version b/ap-fetch-version
index aaf884d..7cc2d98 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -69,6 +69,14 @@ linuxfirmware)
repo_tree_rev_fetch_git linuxfirmware \
$UPSTREAM_TREE_LINUXFIRMWARE master daily-cron.$branch
;;
+libvirt)
+ repo_tree_rev_fetch_git libvirt \
+ $TREE_LIBVIRT master $LOCALREV_LIBVIRT
+ ;;
+gnulib-libvirt)
+ repo_tree_rev_fetch_git gnulib-libvirt \
+ $TREE_GNULIB_LIBVIRT master $LOCALREV_GNULIB_LIBVIRT
+ ;;
osstest)
if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
git fetch $HOME/testing.git pretest:ap-fetch >&2
diff --git a/ap-fetch-version-old b/ap-fetch-version-old
index 21ff786..3879e09 100755
--- a/ap-fetch-version-old
+++ b/ap-fetch-version-old
@@ -27,6 +27,7 @@ select_xenbranch
: ${BASE_TAG_LINUX2639:=tested/2.6.39.x}
: ${BASE_LOCALREV_LINUX:=daily-cron.$branch.old}
+: ${BASE_LOCALREV_LIBVIRT:=daily-cron.$branch.old}
: ${BASE_TREE_QEMU_UPSTREAM:=${TREE_QEMU_UPSTREAM/\/staging\//\/}}
@@ -73,6 +74,14 @@ linuxfirmware)
repo_tree_rev_fetch_git linuxfirmware \
$TREE_LINUXFIRMWARE master daily-cron-old.$branch
;;
+libvirt)
+ repo_tree_rev_fetch_git libvirt \
+ $BASE_TREE_LIBVIRT xen-tested-master $BASE_LOCALREV_LIBVIRT
+ ;;
+gnulib-libvirt)
+ # No push gate, same as ap-fetch-version
+ ./ap-fetch-version $branch
+ ;;
osstest)
if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
git fetch -f $HOME/testing.git incoming:ap-fetch
diff --git a/ap-print-url b/ap-print-url
index f2812d0..b1af0b9 100755
--- a/ap-print-url
+++ b/ap-print-url
@@ -46,6 +46,12 @@ linux-2.6.39)
linuxfirmware)
echo $UPSTREAM_TREE_LINUXFIRMWARE
;;
+libvirt)
+ echo $TREE_LIBVIRT
+ ;;
+gnulib-libvirt)
+ echo $TREE_GNULIB_LIBVIRT
+ ;;
osstest)
echo none:;
;;
diff --git a/ap-push b/ap-push
index dc0a166..69f88a3 100755
--- a/ap-push
+++ b/ap-push
@@ -32,6 +32,7 @@ select_xenbranch
TREE_LINUX=$PUSH_TREE_LINUX
TREE_QEMU_UPSTREAM=$XENBITS:/home/xen/git/qemu-upstream-${xenbranch#xen-}.git
TREE_XEN=$PUSH_TREE_XEN
+TREE_LIBVIRT=$PUSH_TREE_LIBVIRT
if info_linux_tree "$branch"; then
cd $repos/linux
@@ -72,6 +73,14 @@ linux-xen-*)
cd $repos/linux
git push $TREE_LINUX $revision:refs/heads/$branch
;;
+libvirt)
+ cd $repos/libvirt
+ git push $TREE_LIBVIRT $revision:xen-tested-master
+ ;;
+gnulib-libvirt)
+ # No gate
+ echo "gnulib-libvirt has not push gate, refusing to push" >&2
+ exit 1
osstest)
git push $HOME/testing.git $revision:incoming
git push $XENBITS:/home/xen/git/osstest.git $revision:master
diff --git a/cr-daily-branch b/cr-daily-branch
index 02fef15..05fdee1 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -133,6 +133,14 @@ if [ "x$REVISION_LINUX_ARM" = x ]; then
export REVISION_LINUX_ARM
fi
fi
+if [ "x$REVISION_LIBVIRT" = x ]; then
+ determine_version REVISION_LIBVIRT libvirt
+ export REVISION_LIBVIRT
+fi
+if [ "x$REVISION_GNULIB_LIBVIRT" = x ]; then
+ determine_version REVISION_GNULIB_LIBVIRT gnulib-libvirt
+ export REVISION_GNULIB_LIBVIRT
+fi
if [ "x$REVISION_LINUXFIRMWARE" = x ]; then
determine_version REVISION_LINUXFIRMWARE linuxfirmware
export REVISION_LINUXFIRMWARE
@@ -165,6 +173,10 @@ linuxfirmware)
realtree=linuxfirmware
NEW_REVISION=$REVISION_LINUXFIRMWARE
;;
+libvirt)
+ realtree=libvirt
+ NEW_REVISION=$REVISION_LIBVIRT
+ ;;
*)
NEW_REVISION=''
wantpush=false
diff --git a/cri-common b/cri-common
index 497d4e3..f472927 100644
--- a/cri-common
+++ b/cri-common
@@ -42,6 +42,7 @@ select_xenbranch () {
linux) tree=linux; xenbranch=xen-unstable ;;
linux-*) tree=linux; xenbranch=xen-unstable ;;
linuxfirmware) tree=linuxfirmware; xenbranch=xen-unstable ;;
+ libvirt) tree=libvirt; xenbranch=xen-unstable ;;
osstest) tree=osstest; xenbranch=xen-unstable ;;
esac
if [ "x$tree" = xlinux ]; then
diff --git a/make-flight b/make-flight
index 92b813a..aadf58b 100755
--- a/make-flight
+++ b/make-flight
@@ -64,6 +64,9 @@ job_create_test_filter_callback () {
return 1;;
esac
;;
+ libvirt)
+ if [ x$toolstack != xlibvirt ] ; then return 1; fi
+ ;;
*)
case "$job" in
*-qemuu-*)
@@ -238,6 +241,10 @@ test_matrix_do_one () {
$xenarch $dom0arch \
$debian_runvars all_hostflags=$most_hostflags
+ job_create_test test-$xenarch$kern-$dom0arch-libvirt test-debian libvirt \
+ $xenarch $dom0arch \
+ $debian_runvars all_hostflags=$most_hostflags
+
# No further arm tests at the moment
if [ $dom0arch = armhf ]; then
return
diff --git a/mfi-common b/mfi-common
index 598aa02..235575e 100644
--- a/mfi-common
+++ b/mfi-common
@@ -162,6 +162,18 @@ create_build_jobs () {
tree_linuxfirmware=$TREE_LINUXFIRMWARE \
revision_linuxfirmware=$REVISION_LINUXFIRMWARE
+ ./cs-job-create $flight build-$arch-libvirt build-libvirt \
+ arch=$arch \
+ tree_xen=$TREE_XEN \
+ $RUNVARS $BUILD_RUNVARS $BUILD_LIBVIRT_RUNVARS $arch_runvars \
+ $suite_runvars \
+ host_hostflags=$build_hostflags \
+ buildjob=${bfi}build-$arch \
+ tree_libvirt=$TREE_LIBVIRT revision_libvirt=$REVISION_LIBVIRT\
+ ${TREEVCS_LIBVIRT:+treevcs_libvirt=}${TREEVCS_LIBVIRT} \
+ tree_gnulib_libvirt=$TREE_GNULIB_LIBVIRT
revision_gnulib_libvirt=$REVISION_GNULIB_LIBVIRT\
+
${TREEVCS_GNULIB_LIBVIRT:+treevcs_gnulib_libvirt=}${TREEVCS_GNULIB_LIBVIRT}
+
case "$arch" in
armhf) continue;; # don't do any other kernel builds
esac
@@ -193,6 +205,7 @@ job_create_test () {
xenbuildjob="${bfi}build-$xenarch"
buildjob="${bfi}build-$dom0arch"
+ tsbuildjob=
case "$xenbranch:$toolstack" in
xen-3.*-testing:*) ;;
@@ -203,11 +216,13 @@ job_create_test () {
*:xend) xenbuildjob="$xenbuildjob-xend"
buildjob="${bfi}build-$dom0arch-xend"
;;
+ *:libvirt) tsbuildjob="libvirtbuildjob=$buildjob-libvirt"
+ ;;
esac
./cs-job-create $flight $job $recipe toolstack=$toolstack \
$RUNVARS $TEST_RUNVARS $most_runvars \
- xenbuildjob=$xenbuildjob buildjob=$buildjob "$@"
+ xenbuildjob=$xenbuildjob buildjob=$buildjob $tsbuildjob "$@"
}
# Iterate over xenarch, dom0arch and kernel calling test_matrix_do_one
diff --git a/sg-run-job b/sg-run-job
index d336fbd..3d44331 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -312,6 +312,7 @@ proc test-guest-nomigr {g} {
proc need-hosts/build {} { return BUILD }
proc need-hosts/build-kern {} { return BUILD }
+proc need-hosts/build-libvirt {} { return BUILD }
proc run-job/build {} {
run-ts . = ts-xen-build
@@ -321,6 +322,10 @@ proc run-job/build-kern {} {
run-ts . = ts-kernel-build
}
+proc run-job/build-libvirt {} {
+ run-ts . = ts-libvirt-build
+}
+
proc prepare-build-host {} {
global jobinfo
run-ts broken = ts-hosts-allocate + host
diff --git a/standalone b/standalone
index 8d9d88a..6b08995 100755
--- a/standalone
+++ b/standalone
@@ -171,7 +171,7 @@ case $op in
job=$1; shift
- for d in '' xen kern ; do
+ for d in '' xen kern libvirt ; do
runvar="path_${d}dist"
path="build/${d}dist.tar.gz"
if [ -f "logs/$flight/$job/$path" ] ; then
diff --git a/ts-libvirt-build b/ts-libvirt-build
new file mode 100755
index 0000000..9c731ce
--- /dev/null
+++ b/ts-libvirt-build
@@ -0,0 +1,162 @@
+#!/usr/bin/perl
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2009-2014 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+our $ho= selecthost($whhost);
+
+my $leaf= "build.$flight.$job";
+my $homedir = get_host_property($ho, 'homedir', '/home/osstest');
+my $builddir= "$homedir/$leaf";
+my $xendist= "$builddir/xendist";
+
+sub libvirtd_init ();
+
+my $makeflags= get_host_property($ho, 'build make flags', '-j4');
+
+sub checkout () {
+ target_cmd($ho, "rm -rf $builddir && mkdir $builddir && mkdir $xendist",
600);
+
+ my $path = get_stashed("path_dist", $r{"buildjob"});
+ my $distcopy= "$builddir/dist.tar.gz";
+ target_putfile($ho, 300, $path, $distcopy);
+ target_cmd($ho, "tar -C $xendist -hzxf $distcopy", 300);
+
+ build_clone($ho, 'libvirt', $builddir, 'libvirt');
+ build_clone($ho, 'gnulib_libvirt', $builddir, 'gnulib-libvirt');
+}
+
+sub config() {
+ # Uses $GNULIB_SRCDIR because ./autogen.sh doesn't propagate
+ # --gnulib-srcdir to ./bootstap.
+ target_cmd_build($ho, 3600, $builddir, <<END);
+ cd libvirt
+ CFLAGS="-I$xendist/usr/local/include/" \\
+ LDFLAGS="-L$xendist/usr/local/lib/
-Wl,-rpath-link=$xendist/usr/local/lib/" \\
+ GNULIB_SRCDIR=$builddir/gnulib-libvirt \\
+ ./autogen.sh --no-git \\
+ --with-libxl --without-xen --without-xenapi
--without-selinux \\
+ --without-lxc --without-vbox --without-uml \\
+ --sysconfdir=/etc --localstatedir=/var #/
+END
+}
+
+sub build() {
+ target_cmd_build($ho, 3600, $builddir, <<END);
+ cd libvirt
+ (make $makeflags 2>&1 && touch ../build-ok-stamp) |tee ../log
+ test -f ../build-ok-stamp #/
+ echo ok.
+END
+}
+
+sub install() {
+ target_cmd_build($ho, 300, $builddir, <<END);
+ mkdir -p dist
+ cd libvirt
+ make $makeflags install DESTDIR=$builddir/dist
+ mkdir -p $builddir/dist/etc/init.d
+END
+ target_putfilecontents_stash($ho, 60,
+ libvirtd_init(),
+ $builddir.'/dist/etc/init.d/libvirtd');
+ target_cmd_build($ho, 60, $builddir, <<END);
+ chmod +x $builddir/dist/etc/init.d/libvirtd
+END
+}
+
+sub collectversion_gnulib () {
+ store_revision($ho, 'gnulib_libvirt', "$builddir/gnulib-libvirt", 1);
+}
+
+checkout();
+config();
+build();
+install();
+collectversion_gnulib();
+built_stash($ho, $builddir, 'dist', 'libvirtdist');
+
+sub libvirtd_init () {
+ return <<'END';
+#! /bin/sh
+#
+# Init script for libvirtd
+#
+# Loosely based on Debian libvirt-bin initscript:
+# (c) 2007 Guido Guenther <agx@xxxxxxxxxxx>
+# based on the skeletons that comes with dh_make
+#
+### BEGIN INIT INFO
+# Provides: libvirt-bin libvirtd
+# Required-Start: $network $local_fs $remote_fs $syslog
+# Required-Stop: $local_fs $remote_fs $syslog
+# Should-Start: avahi-daemon cgconfig
+# Should-Stop: avahi-daemon cgconfig
+# Default-Start: 2 3 4 5
+# Default-Stop: 0 1 6
+# Short-Description: libvirt management daemon
+### END INIT INFO
+
+PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+export PATH
+DAEMON=/usr/local/sbin/libvirtd
+NAME=libvirtd
+DESC="libvirt management daemon"
+
+test -x $DAEMON || exit 0
+. /lib/lsb/init-functions
+
+PIDFILE=/var/run/$NAME.pid
+
+case "$1" in
+ start)
+ [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
+ start-stop-daemon --start --pidfile $PIDFILE \
+ --exec $DAEMON -- -d $libvirtd_opts
+ [ "$VERBOSE" != no ] && log_end_msg $?
+ ;;
+ stop)
+ [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
+ start-stop-daemon --stop --pidfile $PIDFILE \
+ --exec $DAEMON
+ [ "$VERBOSE" != no ] && log_end_msg $?
+ ;;
+ restart)
+ stop
+ start
+ ;;
+ reload|force-reload)
+ start-stop-daemon --stop --signal 1 --quiet --pidfile \
+ /var/run/$NAME.pid --exec $DAEMON
+ ;;
+ *)
+ N=/etc/init.d/libvirtd
+ echo "Usage: $N {start|stop|restart|reload|force-reload}" >&2
+ exit 1
+ ;;
+esac
+
+exit 0
+END
+}
diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 528d0a4..a4e166e 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -172,6 +172,9 @@ sub prep () {
libx11-dev git-core uuid-dev gettext gawk
libsdl-dev libyajl-dev libaio-dev
libpixman-1-dev
libglib2.0-dev pkg-config
+ autoconf automake libtool xsltproc
+ libxml2-utils libxml2-dev libnl-dev
+ libdevmapper-dev w3c-dtd-xhtml
ccache));
target_cmd_root($ho, "chmod -R a+r /usr/share/git-core/templates");
diff --git a/ts-xen-install b/ts-xen-install
index 4a810e4..ad0254c 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -60,10 +60,15 @@ sub packages () {
if ($r{arch} eq 'i386') {
target_install_packages($ho, 'libc6-xen');
}
+ target_install_packages($ho, @{toolstack()->{ExtraPackages}})
+ if toolstack()->{ExtraPackages};
}
sub extract () {
- foreach my $part ('', 'kern', 'xen') {
+ my @parts = ('', 'kern', 'xen');
+ push @parts, 'libvirt' if $r{toolstack} eq "libvirt";
+
+ foreach my $part (@parts) {
target_extract_jobdistpath($ho, $part, "path_${part}dist",
$r{"${part}buildjob"}, \%distpath);
}
--
1.9.0
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |