Re: [Xen-devel] [OSSTEST Nested PATCH v11 6/7] Compose the main recipe of nested test job

["Pang, LongtaoX" <longtaox.pang@xxxxxxxxx>]

> -----Original Message-----
> From: Ian Campbell [mailto:ian.campbell@xxxxxxxxxx]
> Sent: Monday, June 15, 2015 5:08 PM
> To: Hu, Robert
> Cc: Ian Jackson; Pang, LongtaoX; xen-devel@xxxxxxxxxxxxx; wei.liu2@xxxxxxxxxx
> Subject: Re: [OSSTEST Nested PATCH v11 6/7] Compose the main recipe of nested
> test job
> 
> On Sun, 2015-06-14 at 20:51 +0800, Robert Hu wrote:
> > On Fri, 2015-06-12 at 16:27 +0100, Ian Jackson wrote:
> > > Pang, LongtaoX writes ("RE: [OSSTEST Nested PATCH v11 6/7] Compose the
> main recipe of nested test job"):
> > > > > From: Ian Jackson [mailto:Ian.Jackson@xxxxxxxxxxxxx]
> > > ...
> > > > > leak-check compares the set of objects present at the `leak-check
> > > > > check' step with the set of objects present at the `basis' step, and
> > > > > the check fails if there are any new objects.  For this purpose,
> > > > > objects includes domains, corefiles, etc.
> > > > >
> > > > OK, so the recipe in sg-run-job should be like below, please correct me 
> > > > if
> something wrong.
> > > > proc need-hosts/test-nested {} {return host}
> > > > proc run-job/test-nested {} {
> > >
> > > This is roughly right, but thinking about it, you want ts-logs-capture
> > > to run even if the previous steps fail.
> > >
> > > I think it might be better to reuse (subvert?) the existing machinery
> > > in sg-run-job, by adding the l1 to need_xen_hosts.
> > >
> > > Maybe something like
> > >
> > >   proc add-xen-host-retrospectively {ident} {
> > >       global need_xen_hosts
> > >       ts-leak-check $ident + basis
> > >       lappend need_xen_hosts $ident
> > >   }
> > >
> > > ?
> > >
> > > And then call
> > >
> > >   add-xen-host-retrospectively l1
> > >
> > > at the appropriate point.
> > Thanks Ian J.. Since I'm not familiar with tcl and your sg-run-job
> > framework, does here 'appropriate point' refers to before
> >     per-host-ts .       =(*)             {ts-leak-check basis}
> > in proc run-job {job}? but then l1 doesn't exist yet I'm afraid.
> > If after that point, the l1 has missd check basis step.
> 
> Note that Ian included a ts-leak-check in his
> add-xen-host-retrospectively function, so you needn't worry about this,
> you should do it jsut after the L1 has booted into Xen, I think.
> 
> What you get automatically here is the final leak check (i.e. the
> comparison against the basis).
Thanks Ian C.
So, I modify the nested job recipe in sg-run-job as below.
proc add-xen-host-retrospectively {ident} {
    global need_xen_hosts
    run-ts . = ts-leak-check basis $ident
    lappend need_xen_hosts $ident
}

proc need-hosts/test-nested {} {return host}
proc run-job/test-nested {} {
    run-ts . = ts-debian-hvm-install + host nestedl1
    run-ts . = ts-nested-setup + host nestedl1
    run-ts . = ts-xen-install nestedl1
    run-ts . = ts-host-reboot nestedl1
    add-xen-host-retrospectively nestedl1
    run-ts . = ts-debian-hvm-install nestedl1 nestedl2
    run-ts . = ts-guest-stop nestedl1 nestedl2
    run-ts . = ts-guest-destroy + host nestedl1
}

After executing command ' ./standalone run-job --simulate -h dummy 
test-amd64-amd64-qemuu-nested | grep testid',
get below information:
root@OSSTEST:~/v11_pretest_3/osstest_v11# ./standalone run-job --simulate -h 
dummy test-amd64-amd64-qemuu-nested | grep testid                
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 1 
testid build-check(1) ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 2 
testid hosts-allocate ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 3 
testid host-install(3) ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 4 
testid host-ping-check-native ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 5 
testid xen-install ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 6 
testid xen-boot ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 7 
testid host-ping-check-xen ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 8 
testid leak-check/basis(8) ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 9 
testid debian-hvm-install ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 10 
testid nested-setup ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 11 
testid xen-install/nestedl1 ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 12 
testid host-reboot/nestedl1 ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 13 
testid leak-check/basis/nestedl1 ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 14 
testid debian-hvm-install/nestedl1/nestedl2 ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 15 
testid guest-stop/nestedl1/nestedl2 ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 16 
testid guest-destroy ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 17 
testid leak-check/check ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 18 
testid leak-check/check/nestedl1 ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 19 
testid capture-logs(19) ==========
2015-06-17 07:33:35 Z standalone.test-amd64-amd64-qemuu-nested ========== 20 
testid capture-logs/nestedl1(20) ==========
root@OSSTEST:~/v11_pretest_3/osstest_v11#

But, for testid of '18', I think it will failed to execute 
'leak-check/check/nestedl1', since 
'nestedl1' has been destroyed via the action of 'run-ts . = ts-guest-destroy + 
host nestedl1'. 
Please correct me if I am wrong.

Another question, after execute testid of '13'(leak-check/basis/nestedl1), the 
testing will fail and exit with below message.
Something wrong with my test environment?
2015-06-17 08:10:25 Z executing ssh ... root@xxxxxxxxxxxxxx xl list
2015-06-17 08:10:25 Z executing ssh ... root@xxxxxxxxxxxxxx ps -wwef
2015-06-17 08:10:25 Z executing ssh ... root@xxxxxxxxxxxxxx ps -wwef
2015-06-17 08:10:25 Z executing ssh ... root@xxxxxxxxxxxxxx xenstore-ls -fp
2015-06-17 08:10:26 Z executing ssh ... root@xxxxxxxxxxxxxx find /tmp /var/run 
/var/tmp /var/lib/xen /var/core ! -type d -print0 -ls -printf '\0'
find: `/var/core': No such file or directory
2015-06-17 08:10:26 Z command nonzero waitstatus 256: timeout 60 ssh -o 
StrictHostKeyChecking=no -o BatchMode=yes -o ConnectTimeout=100 -o 
ServerAliveInterval=100 -o PasswordAuthentication=no -o 
ChallengeResponseAuthentication=no -o 
UserKnownHostsFile=tmp/t.known_hosts_standalone.test-amd64-amd64-qemuu-nested 
root@xxxxxxxxxxxxxx find /tmp /var/run /var/tmp /var/lib/xen /var/core ! -type 
d -print0 -ls -printf '\0'
BROKEN: listing/checking leakable objects: status 256 at Osstest/TestSupport.pm 
line 400.
; marked standalone.test-amd64-amd64-qemuu-nested broken at 
Osstest/TestSupport.pm line 218.
+ rc=2
+ date -u +%Y-%m-%d %H:%M:%S Z exit status 2
2015-06-17 08:10:26 Z exit status 2
+ exit 2
2015-06-17 08:10:26 Z standalone.test-amd64-amd64-qemuu-nested 13 status status 
fail
> 
> > > If you do this then the main run-job proc will automatically do the
> > > leak-check and the logs-capture for you.
> > >
> > >
> > > Thinking about this leads me to ask another question.  Suppose that a
> > > bug causes the l1 to lock up completely.  ts-logs-capture will attempt
> > > to hard reboot a locked-up host.  If it can't fetch any logs, it calls
> > >     target_reboot_hard($ho);
> > >
> > > What will that do if $ho refers to the l1 ?  It relies on the power
> > > method.  Does your nested l1 "host" have a power method ?
> > I'm afraid l1 won't like normal hosts has power cycle operations. Maybe
> > we need to simulate it?
> 
> Perhaps arrange for an appropriate PowerMethod for "hosts which are
> actually guests"?
> 
I think maybe we need to refactor 'power_cycle' function in TestSupport.pm. I 
have not try it, something like below?
sub power_cycle ($) {
    my ($ho) = @_;
+    if (guest_var($ho,"enable_nestedhvm",'') =~ m/true/) {
+        guest_destroy($ho);
+        guest_create($gho);
+        guest_await_dhcp_tcp($gho,300);
+        guest_check_up($gho);
+    } else {
        $mjobdb->host_check_allocated($ho);
        die "refusing to set power state for host $ho->{Name}".
            " possibly shared with other jobs\n"
            if $ho->{SharedMaybeOthers};
        power_state($ho, 0);
        power_cycle_sleep($ho);
        power_state($ho, 1);
+    }
}
> Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel