[Xen-devel] [PATCH OSSTEST] README: Add some core concepts and terminology

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

---
 README | 202 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 201 insertions(+), 1 deletion(-)

diff --git a/README b/README
index 29c9d45..60379c4 100644
--- a/README
+++ b/README
@@ -1,3 +1,197 @@
+Introduction
+============
+
+OSStest is the Xen Project automated test infrastructure.
+
+Terminology
+===========
+
+"flight":
+
+    Each run of osstest is referred to as a "flight". Each flight is
+    given a unique ID (a number or name).
+
+"job":
+
+    Each flight consists of one or more "jobs". These are a sequence
+    of test steps run in order and correspond to a column in the test
+    report grid. They have names like "build-amd64" or
+    "test-amd64-amd64-pv". A job can depend on the output of another
+    job in the flight -- e.g. most test-* jobs depend on one or more
+    build-* jobs.
+
+"step":
+
+    Each job consists of multiple "steps" which is an individual test
+    operation, such as "build the hypervisor", "install a guest",
+    "start a guest", "migrate a guest", etc. A step corresponds to a
+    cell in the results grid. A given step can be reused in multiple
+    different jobs, e.g. the "xen build" step is used in several
+    different build-* jobs. This reuse can be seen in the rows of the
+    results grid.
+
+"runvar":
+
+    A runvar is a named textual variable associated with each job in a
+    given flight. They serve as both the inputs and outputs to the
+    job.
+
+    For example a Xen build job may have input runvars "tree_xen" (the
+    tree to clone) and "revision_xen" (the version to test). As output
+    it sets "path_xendist" which is the path to the tarball of the
+    resulting binary.
+
+    As a further example a test job may have an input runvar
+    "xenbuildjob" which specifies which build job produced the binary
+    to be tested. The "xen install" step can then read this runvar in
+    order to find the binary to install.
+
+    Other runvars also exist covering things such as:
+
+        * constraints on which machines in the test pool a job can be
+          run on (e.g. the architecure, the need for a particular
+          processor class, the presence of SR-IOV etc).
+
+        * the parameters of the guest to test (e.g. distro, PV vs HVM
+          etc).
+
+Operation
+=========
+
+A flight is constructed by the "make-flight" script.
+
+"make-flight" will allocate a new flight number, create a set of jobs
+with input runvars depending on the configuration (e.g. branch/version
+to test).
+
+A flight is run by the "mg-execute-flight" script, which in turn calls
+"sg-execute-flight". "sg-execute-flight" then spawns an instance of
+"sg-run-job" for each job in the flight.
+
+"sg-run-job" encodes various recipes (sequences of steps) which are
+referenced by each job's configuration. It then runs each of these in
+turn, taking into account the prerequisites etc, by calling the
+relevant "ts-*" scripts.
+
+When running in standalone mode it is possible to run any of these
+steps by hand, ("mg-execute-flight", "sg-run-job", "ts-*") although
+you will need to find the correct inputs (some of which are documented
+below) and perhaps take care of prerequisites yourself (e.g. running
+"./sg-run-job test-armhf-armhf-xl" means you must have done
+"./sg-runjob build-armhf" and "build-armhf-pvops" first.
+
+Results
+=======
+
+For flights run automatically by the infrastructure an email report is
+produced. For most normal flights this is mailed to the xen-devel
+mailing list. The report for flight 24438 can be seen at
+
+    http://lists.xen.org/archives/html/xen-devel/2014-01/msg01614.html
+
+The report will link to a set of complete logs. Since these are
+regularly expired due to space constraints the logs for flight 24438
+have been archived to
+
+    http://xenbits.xen.org/docs/osstest-output-example/24438/
+
+NB: to save space any files larger than 100K have been replaced with a
+placeholder.
+
+The results grid contains an overview of the flight's execution.
+
+The results for each job are reached by clicking the header of the
+column in the results grid which will lead to reports such as:
+
+    
http://xenbits.xen.org/docs/osstest-output-example/24438/test-amd64-amd64-xl/info.html
+    
http://xenbits.xen.org/docs/osstest-output-example/24438/build-amd64/info.html
+
+The job report contains all of the logs and build outputs associated
+with this job.
+
+The logs for a step are reached by clicking the individual cells of
+the results grid, or by clicking the list of steps on the job
+report. In either case this will lead to a report such as
+
+    
http://xenbits.xen.org/docs/osstest-output-example/24438/test-amd64-amd64-xl/4.ts-xen-install.log
+
+Additional details (e.g. serial logs, guest cfg files, etc) will be
+available in the complete logs associated with the containing job.
+
+The runvars are listed in the results page for the job as "Test
+control variables". e.g. See the end of:
+
+    
http://xenbits.xen.org/docs/osstest-output-example/24438/test-amd64-amd64-xl/info.html
+
+In order to find the binaries which went into a test job you should
+consult the results page for that job and find the relevant build
+job. e.g.
+
+    
http://www.chiark.greenend.org.uk/~xensrcts/logs/24438/test-amd64-amd64-xl/info.html
+
+lists "xenbuildjob" as "build-amd64". Therefore the input binaries are
+found at
+
+    
http://www.chiark.greenend.org.uk/~xensrcts/logs/24438/build-amd64/info.html
+
+which is linked from the top of the relevant column in the overview
+grid.
+
+Script Naming Conventions
+=========================
+
+Most of the scripts follow a naming convention:
+
+ap-*:   Adhoc push scripts
+
+cr-*:   Cron scripts
+
+cri-*:  Cron scripts (internal)
+
+cs-*:   Control Scripts
+
+mg-*:   Management scripts
+
+ms-*:   Management Services
+
+sg-*:   ?
+
+ts-*:   Test Step scripts.
+
+Jobs
+====
+
+The names of jobs follow some common patterns:
+
+    build-$ARCH
+
+        Build Xen for $ARCH
+
+    build-$ARCH-xend
+
+        Build Xen for $ARCH, with xend enabled
+
+    build-$ARCH-pvops
+
+        Build an upstream ("pvops") kernel for $ARCH
+
+build-$ARCH-oldkern
+
+        Build the old "linux-2.6.18-xen" tree for $ARCH
+
+test-$XENARCH-$DOM0ARCH-<CASE>
+
+        A test <CASE> running a $XENARCH hypervisor with a $DOM0ARCH
+        dom0.
+
+        Some tests also have a -$DOMUARCH suffix indicating the
+        obvious thing.
+
+NB: $ARCH (and $XENARCH etc) are Debian arch names, i386, amd64, armhf.
+
+Standalone Mode
+===============
+
 To run osstest in standalone mode:
 
  - You need to install
@@ -18,7 +212,7 @@ To run osstest in standalone mode:
    gives you the "branch" consisting of tests run for the xen-unstable
    push gate.  You need to select a job.  The list of available jobs
    is that shown in the publicly emailed test reports on xen-devel, eg
-     http://lists.xen.org/archives/html/xen-devel/2013-08/msg02529.html
+     http://lists.xen.org/archives/html/xen-devel/2014-01/msg01614.html
 
    If you don't want to repro one of those and don't know how to
    choose a job, choose one of
@@ -26,6 +220,12 @@ To run osstest in standalone mode:
 
  - Run ./standalone-reset
 
+   This will call "make-flight" for you to create a flight targetting
+   xen-unstable (this can be adjusted by passing parameters to
+   standalone-reset). By default the flight identifier is
+   "standalone". standalone-reset will also make sure that certain
+   bits of static data are available (e.g. Debian installer images)
+
  - Then you can run
       ./sg-run-job <job>
    to run that job on the default host.  NB in most cases this will
-- 
1.8.5.2


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel