Re: [XEN PATCH v2 1/3] CI: Rework run-tools-test exit path

[Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>]

On Wed, Jul 30, 2025 at 05:26:00PM +0200, Anthony PERARD wrote:
> From: Anthony PERARD <anthony.perard@xxxxxxxxxx>
> 
> The main script expect to find the string "$passed" or it just timeout
> and doesn't try to download the junit file in this case. So we ignore
> the return value of run-tools-test to always print "$passed" and
> instead look for failure in the generated junit file. If the junit
> report is incomplete, this will also result in a failure of the job.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@xxxxxxxxxx>
> ---
> 
> Notes:
>     Changes in v2:
>     - This squash both patch "CI: Ignore run-tools-test return value" and
>       "CI: Have the gitlab job fail on tools/tests failure"
>     - grep for '<failure type="failure"' instead of '</failure>'
> 
>  automation/scripts/qubes-x86-64.sh | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/automation/scripts/qubes-x86-64.sh 
> b/automation/scripts/qubes-x86-64.sh
> index 2750d24eba..21dcd9b063 100755
> --- a/automation/scripts/qubes-x86-64.sh
> +++ b/automation/scripts/qubes-x86-64.sh
> @@ -135,10 +135,11 @@ done
>      ### tests: tools-tests-pv, tools-tests-pvh
>      "tools-tests-pv"|"tools-tests-pvh")
>          retrieve_xml=1
> -        passed="test passed"
> +        passed="run-tools-test over"
>          domU_check=""
>          dom0_check="
> -/root/run-tools-tests /usr/lib/xen/tests /tmp/tests-junit.xml && echo 
> \"${passed}\"
> +/root/run-tools-tests /usr/lib/xen/tests /tmp/tests-junit.xml ||:
> +echo \"${passed}\"
>  nc -l -p 8080 < /tmp/tests-junit.xml >/dev/null &
>  "
>          if [ "${test_variant}" = "tools-tests-pvh" ]; then
> @@ -297,6 +298,14 @@ TEST_RESULT=$?
>  
>  if [ -n "$retrieve_xml" ]; then
>      nc -w 10 "$SUT_ADDR" 8080 > tests-junit.xml </dev/null
> +    # Findout if one of the test failed
> +    if ! grep -q '</testsuites>' tests-junit.xml; then
> +        echo "ERROR: tests-junit.xml is incomplete or missing."
> +        TEST_RESULT=1
> +    # Only match "type=failure" to allow to "tolerable" for example.
> +    elif grep -q '<failure type="failure"' tests-junit.xml; then

Maybe drop -q here, or add some message? Otherwise it's not obvious why it 
failed. See for example:
https://gitlab.com/xen-project/people/marmarek/xen/-/jobs/10968674458

Yes, you can click "Test summary" link, but it isn't obvious by looking
at the final part of the log. Alternatively, this can be combined with
my "CI: list failed tests at the end of tools job" patch (already
reviewed by Andrew), but I had an impression this series is supposed to
be included instead of that one.

> +        TEST_RESULT=1
> +    fi
>  fi
>  
>  exit "$TEST_RESULT"
> -- 
> Anthony PERARD
> 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

Attachment: signature.asc
Description: PGP signature