Re: [Xen-devel] [PATCH XTF 3/3] xtf-runner: regularise runner exit code

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 22/07/16 10:29, Wei Liu wrote:

> On Thu, Jul 21, 2016 at 08:04:59PM +0100, Andrew Cooper wrote:
> > On 21/07/2016 16:44, Wei Liu wrote:
> > > Report the first "ERROR" and "FAILURE" if found, otherwise report "SKIP"
> > > if found. Eventually if everything is ok the exit code will be 0.
> > >
> > > See runner code for numeric exit code space.
> > >
> > > Signed-off-by: Wei Liu <wei.liu2@xxxxxxxxxx>
> > > ---
> > >   xtf-runner | 12 +++++++++---
> > >   1 file changed, 9 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/xtf-runner b/xtf-runner
> > > index 17ce933..ebe5c27 100755
> > > --- a/xtf-runner
> > > +++ b/xtf-runner
> > > @@ -249,17 +249,23 @@ def run_tests(args):
> > >       if not len(tests):
> > >           raise RunnerError("No tests to run")
> > >   
> > > -    rc = 0
> > > +    rc = exit_code('SUCCESS')
> > This logic would be easier to express if you use the indices of
> > all_results as a measure of severity.
> >
> > e.g.
> >
> > rc = all_results.index('SUCCESS')
> >
> > >       results = []
> > >   
> > >       for test in tests:
> > >   
> > >           res = run_test(test)
> > > -        if res != "SUCCESS":
> > > -            rc = 1
> > > +        if res in ("ERROR", "FAILURE") and rc == exit_code('SUCCESS'):
> > > +            rc = exit_code(res)
> > res_idx = all_results.index(res)
> > if res_idx > rc:
> >      rc = res_idx
> I intended to report the first "error" or "failure" encountered.
>
> This would cause a FAILURE to overwrite previous ERROR result. Is that
> what you want?
When running more than one test, the overall result should be the most 
severe.  So yes, a subsequent FAILURE should override an ERROR.
~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel