RE: [Xen-devel] [PATCH] Make Xend use consoled and xc_console

["Ian Pratt" <m+Ian.Pratt@xxxxxxxxxxxx>]

> > Your regression test sounds useful, I'll likely be needing 
> it soon...
> 
> Trying to figure out how best to submit it to the tree.  The 
> problem is that it requires an initrd that can be made to 
> execute a command during startup.  I'll probably post the 
> code with just a readme in the next couple hours.

Excellent. Having more regression tests is always useful.
 
> Would be nice to have a general way to do this for testsuites 
> in Xen though.  Any chance we'll see XenBench2 soon?

Yep, soon now -- it seems to be coming together nicely. The first
priority is to 'go live' with the web display matrix and start filling
in results, and then to get it packaged and 'documented' so that others
can run it and start adding tests etc. I've appended the first test
programme that we're aiming for (I don't think we'll quite have
everything by rev 1, but quite a lot of it)

The infrastructure is capable of taking a virgin machine and installing
everything needed to run a given test programme, co-ordinating the
tests, and then uploading the results.


Best,
Ian


phase 1: domain 0 uniprocessor tests:
 * lmbench, just the OS tests
 * 'make -j8' build of e.g. linux 2.6.11 with the defualt config
 * run the Linux Test Project (LTP) suite
 * run ttcp rx/tx bw tests to the supervisor, with 552 and 1500 byte
MTUs
 * run udp ping-pong msg latency tests for a few message sizes 64, 1024,
64k
 * run timed 'dd' test to local disks to establish raw disk performance
 * start the control tools
 * start a new VM
 * (NB: ensure all domains in these tests are configured with swap
space)

Phase 2: domain 1 uniprocessor tests:
 * lmbench, just the OS tests
 * 'make -j8' timed build of e.g. linux 2.6.11 with the defualt config
 * run the Linux Test Project (LTP) suite
 * run ttcp rx/tx tests to the supervisor, with 552 and 1500 byte MTUs
 * run udp ping-pong msg latency tests for a few message sizes 64, 1024,
64k
 * run timed 'dd' read/write tests to local disks e.g. of several GB of
data
 * run osdl reaim benchmark
 * run postmark benchmark
 * run tbench benchmark
 * run dbench benchmark
 * run 'crashme' for a few minutes
 * run memtest-0.0.4 suite
 * run LTP in parallel stress test mode for e.g. 15mins
 * run PostgreSQL driven by osdb (this is slightly hard, but we have
experince)
 * 'make -j' max parallel build of e.g. linux 2.6.11 with the defualt
config
 * possibly run SPEC JBB2000 

Phase 3: inter-domain network testing
 * (dom0 to create another VM )
 * run tcp bandwidth tests between VMs using ttcp, 1500 byte MTU
 * run udp ping-pong latency tests between the two VMs, different msg
sizes
 * possibly run the fixed ACE tests between two domains
 
Phase 4: domain 2 SMP tests:
 * (domain 0 to create new SMP VM)
 * repeat the tests that were used on the uniprocessor VM

Phase 5: stress testing
 * (dom0 to create several other SMP VMs, 2-3x over commit on physcial
CPUs )
 * (for these tests we don't care about the benchmark results)
 * in a loop in each VM (including dom0), select benchmarks to run in a
random order.
 * it would be ideal if we could have two domains that run the
inter-domain network tests throughout this phase (i.e. not participating
in the random tests)
 * we need to continue to monitor the liveness of the benchmarks and of
the VMs
 * after running this phase for an hour or two, move to phase 6

Phase 6: stress testing combined with tools testing
 * stop the random benchmark run in domain 0 but let other VMs continue
 * run the following tests for a few minutes each:
   * go through adding VCPUs to each domain, one at a time, sleep 5
   * go through removing VCPUs to each domain, one at a time, sleep 5
   * (number of VCPUs for each domain should be restored)
   * pick two domains. Use the balloon driver to increase the memory of
one 
       and reduce the memory of the other. sleep 10. repeat for several
minutes
   * pick a domain, do a 'xm migrate -l DOMID localhost'
       (NB you must have some slack memory on the machine to do this)
       check migration completes OK, repeat many times
   * (we could add other tests to check other 'xm' commands (e.g.
pincpu)
       but this suite will already get most of the hard cases
   * issue xend restart

Phase 7: shutdown
 * shut down each domU, then shutdown dom0
         

Tests to be run on x86_32, x86_32p and x86_64 builds of Xen / Linux, on
a range of different machines and distros. x86_64 should use a
combination of 32 and 64 bit file systems.




_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel