[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[PATCH v2 2/2] automation: use expect utility in xilinx tests


  • To: <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • From: <victorm.lira@xxxxxxx>
  • Date: Thu, 29 Aug 2024 15:34:23 -0700
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0)
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bDjxXa+T088VquLWdlKJWLM1PWaU1rdeMmmnf3wCXPE=; b=D5b/UW8TX50rgXG4ZbjUQYD2fWb3E/Vm/9R9LRnGprbTT6GgJoXlYOuu2WbtONvS5GiiP9xXB4Wi8X/vm9RI9EWvivtR89T8Cg+4XWQEkDYifC8UY++HMkb4zn9qwj2ZJblg4UqzflF0j7vqpqtxHHJB4OP8Iflal8P2Kv4/0TPgRdLByhPhp4ccYqoebcZnMXB5cYrlFs9g0ZLrLr21glYJbFsIn6w/pQewgf0qBFFu4Ay9oDQgCjX1t7PNApgEYQiQ0KSHj0vYwRIyoCIAwXvuZCGbKdjcI8srtmM+qMQPAUw/uL9I0xmp2SABpso3QNlZSl6yE4+uxQshx1R/YQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kCIclsFLjbTirbjJWJ5DddrjPvcqAim+fjSUpHWaIEph1aBvTwz9Lk0SeijnaDpRTQZ0ZHqZ0o2Vcnc2P9D2d2fPHWCi0sco7dB/HLj8UxwTxy9NsLAfHZZtg5YP7hgDI3txqVl73wW0BjG+EYXpzyyB/6NMNZhNUx7ixPoXdqGuKsXUr5WnhFw+dN0YHIGook70QD00FGMVbSoWWO3x5iNybKWO/4f0IP+eueVaAIdC9WIiioB0LBxmDXil49x4vOuKi3yHhQqNUO3d1kty2xZJ2NvfP6XrV8lk3twIcH8GEEvxJ86bcSHIrHCWxigOmJDhqGPStyTWc4P5z+fU+A==
  • Cc: Victor Lira <victorm.lira@xxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>, Julien Grall <julien@xxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Michal Orzel <michal.orzel@xxxxxxx>
  • Delivery-date: Thu, 29 Aug 2024 22:35:07 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

From: Victor Lira <victorm.lira@xxxxxxx>

Fixes: 95764a0817a5 (automation: update xilinx test scripts (tty))
This patch introduced a CI failure due to a timeout in xilinx-x86_64 test.

Change xilinx-x86_64 and xilinx-arm64 scripts to use "expect" utility
to determine test result and allow early exit from tests.
Add "expect" to xilinx container environment (dockerfile).
Rename references to "QEMU" in "qemu-key.exp" expect script to "TEST" to be
used by both QEMU and hardware tests.

Signed-off-by: Victor Lira <victorm.lira@xxxxxxx>
---
Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
Cc: Jan Beulich <jbeulich@xxxxxxxx>
Cc: Julien Grall <julien@xxxxxxx>
Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>
Cc: Michal Orzel <michal.orzel@xxxxxxx>
Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx
---
 .../build/ubuntu/xenial-xilinx.dockerfile     |  1 +
 automation/gitlab-ci/test.yaml                |  2 ++
 .../scripts/{qemu-key.exp => console.exp}     |  8 +++----
 automation/scripts/qemu-alpine-x86_64.sh      |  6 ++---
 automation/scripts/qemu-smoke-dom0-arm32.sh   |  6 ++---
 automation/scripts/qemu-smoke-dom0-arm64.sh   |  6 ++---
 .../scripts/qemu-smoke-dom0less-arm32.sh      |  6 ++---
 .../scripts/qemu-smoke-dom0less-arm64.sh      |  6 ++---
 automation/scripts/qemu-smoke-ppc64le.sh      |  6 ++---
 automation/scripts/qemu-smoke-riscv64.sh      |  6 ++---
 automation/scripts/qemu-smoke-x86-64.sh       |  6 ++---
 automation/scripts/qemu-xtf-dom0less-arm64.sh |  6 ++---
 .../scripts/xilinx-smoke-dom0-x86_64.sh       | 22 +++++++++----------
 .../scripts/xilinx-smoke-dom0less-arm64.sh    | 19 ++++++++--------
 14 files changed, 54 insertions(+), 52 deletions(-)
 rename automation/scripts/{qemu-key.exp => console.exp} (83%)

diff --git a/automation/build/ubuntu/xenial-xilinx.dockerfile 
b/automation/build/ubuntu/xenial-xilinx.dockerfile
index f03d62e8bd3f..6107d8b7711f 100644
--- a/automation/build/ubuntu/xenial-xilinx.dockerfile
+++ b/automation/build/ubuntu/xenial-xilinx.dockerfile
@@ -20,6 +20,7 @@ RUN apt-get update && \
         git \
         gzip \
         file \
+        expect \
         && \
         apt-get autoremove -y && \
         apt-get clean && \
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 3b339f387fbe..e3ebe3745994 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -84,6 +84,7 @@
   variables:
     CONTAINER: ubuntu:xenial-xilinx
     LOGFILE: qemu-smoke-xilinx.log
+    TEST_TIMEOUT: 120
   artifacts:
     paths:
       - smoke.serial
@@ -103,6 +104,7 @@
     LOGFILE: xilinx-smoke-x86_64.log
     XEN_CMD_CONSOLE: "console=com2 com2=115200,8n1,0x2F8,4"
     TEST_BOARD: "crater"
+    TEST_TIMEOUT: 500
   artifacts:
     paths:
       - smoke.serial
diff --git a/automation/scripts/qemu-key.exp b/automation/scripts/console.exp
similarity index 83%
rename from automation/scripts/qemu-key.exp
rename to automation/scripts/console.exp
index 66c416483146..f538aa6bd06c 100755
--- a/automation/scripts/qemu-key.exp
+++ b/automation/scripts/console.exp
@@ -1,16 +1,16 @@
 #!/usr/bin/expect -f

-if {[info exists env(QEMU_TIMEOUT)]} {
-    set timeout $env(QEMU_TIMEOUT)
+if {[info exists env(TEST_TIMEOUT)]} {
+    set timeout $env(TEST_TIMEOUT)
 } else {
     set timeout 1500
 }

-log_file -a $env(QEMU_LOG)
+log_file -a $env(TEST_LOG)

 match_max 10000

-eval spawn $env(QEMU_CMD)
+eval spawn $env(TEST_CMD)

 expect_after {
     -re "(.*)\r" {
diff --git a/automation/scripts/qemu-alpine-x86_64.sh 
b/automation/scripts/qemu-alpine-x86_64.sh
index 93914c41bc24..1ff689b577e3 100755
--- a/automation/scripts/qemu-alpine-x86_64.sh
+++ b/automation/scripts/qemu-alpine-x86_64.sh
@@ -76,7 +76,7 @@ EOF

 # Run the test
 rm -f smoke.serial
-export QEMU_CMD="qemu-system-x86_64 \
+export TEST_CMD="qemu-system-x86_64 \
     -cpu qemu64,+svm \
     -m 2G -smp 2 \
     -monitor none -serial stdio \
@@ -84,8 +84,8 @@ export QEMU_CMD="qemu-system-x86_64 \
     -device virtio-net-pci,netdev=n0 \
     -netdev user,id=n0,tftp=binaries,bootfile=/pxelinux.0"

-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
 export LOG_MSG="Domain-0"
 export PASSED="BusyBox"

-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-dom0-arm32.sh 
b/automation/scripts/qemu-smoke-dom0-arm32.sh
index 7c282eff3a58..e1cd83880928 100755
--- a/automation/scripts/qemu-smoke-dom0-arm32.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm32.sh
@@ -77,7 +77,7 @@ git clone --depth 1 
https://gitlab.com/xen-project/imagebuilder.git
 bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c config

 rm -f ${serial_log}
-export QEMU_CMD="./qemu-system-arm \
+export TEST_CMD="./qemu-system-arm \
    -machine virt \
    -machine virtualization=true \
    -smp 4 \
@@ -91,8 +91,8 @@ export QEMU_CMD="./qemu-system-arm \
    -bios /usr/lib/u-boot/qemu_arm/u-boot.bin"

 export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 
0x40000000"
-export QEMU_LOG="${serial_log}"
+export TEST_LOG="${serial_log}"
 export LOG_MSG="Domain-0"
 export PASSED="/ #"

-../automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+../automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-dom0-arm64.sh 
b/automation/scripts/qemu-smoke-dom0-arm64.sh
index 81f210f7f50e..4d22a124df11 100755
--- a/automation/scripts/qemu-smoke-dom0-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0-arm64.sh
@@ -93,7 +93,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d 
binaries/ -c binaries/conf

 # Run the test
 rm -f smoke.serial
-export QEMU_CMD="./binaries/qemu-system-aarch64 \
+export TEST_CMD="./binaries/qemu-system-aarch64 \
     -machine virtualization=true \
     -cpu cortex-a57 -machine type=virt \
     -m 2048 -monitor none -serial stdio \
@@ -104,8 +104,8 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \
     -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"

 export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 
0x40000000"
-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
 export LOG_MSG="Domain-0"
 export PASSED="BusyBox"

-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-dom0less-arm32.sh 
b/automation/scripts/qemu-smoke-dom0less-arm32.sh
index 38e8a0b0bd7d..41f6e5d8e615 100755
--- a/automation/scripts/qemu-smoke-dom0less-arm32.sh
+++ b/automation/scripts/qemu-smoke-dom0less-arm32.sh
@@ -130,7 +130,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d . -c 
config

 # Run the test
 rm -f ${serial_log}
-export QEMU_CMD="./qemu-system-arm \
+export TEST_CMD="./qemu-system-arm \
     -machine virt \
     -machine virtualization=true \
     -smp 4 \
@@ -144,8 +144,8 @@ export QEMU_CMD="./qemu-system-arm \
     -bios /usr/lib/u-boot/qemu_arm/u-boot.bin"

 export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 
0x40000000"
-export QEMU_LOG="${serial_log}"
+export TEST_LOG="${serial_log}"
 export LOG_MSG="${dom0_prompt}"
 export PASSED="${passed}"

-../automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+../automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-dom0less-arm64.sh 
b/automation/scripts/qemu-smoke-dom0less-arm64.sh
index ea67650e17da..83e1866ca6c5 100755
--- a/automation/scripts/qemu-smoke-dom0less-arm64.sh
+++ b/automation/scripts/qemu-smoke-dom0less-arm64.sh
@@ -204,7 +204,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d 
binaries/ -c binaries/conf

 # Run the test
 rm -f smoke.serial
-export QEMU_CMD="./binaries/qemu-system-aarch64 \
+export TEST_CMD="./binaries/qemu-system-aarch64 \
     -machine virtualization=true \
     -cpu cortex-a57 -machine type=virt,gic-version=$gic_version \
     -m 2048 -monitor none -serial stdio \
@@ -215,8 +215,8 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \
     -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"

 export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 
0x40000000"
-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
 export LOG_MSG="Welcome to Alpine Linux"
 export PASSED="${passed}"

-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-ppc64le.sh 
b/automation/scripts/qemu-smoke-ppc64le.sh
index 49e189c37058..617096ad1fa8 100755
--- a/automation/scripts/qemu-smoke-ppc64le.sh
+++ b/automation/scripts/qemu-smoke-ppc64le.sh
@@ -10,7 +10,7 @@ machine=$1
 # Run the test
 rm -f ${serial_log}

-export QEMU_CMD="qemu-system-ppc64 \
+export TEST_CMD="qemu-system-ppc64 \
     -bios skiboot.lid \
     -M $machine \
     -m 2g \
@@ -21,7 +21,7 @@ export QEMU_CMD="qemu-system-ppc64 \
     -serial stdio \
     -kernel binaries/xen"

-export QEMU_LOG="${serial_log}"
+export TEST_LOG="${serial_log}"
 export PASSED="Hello, ppc64le!"

-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-riscv64.sh 
b/automation/scripts/qemu-smoke-riscv64.sh
index 422ee03e0d26..8f755d0a6a99 100755
--- a/automation/scripts/qemu-smoke-riscv64.sh
+++ b/automation/scripts/qemu-smoke-riscv64.sh
@@ -5,14 +5,14 @@ set -ex -o pipefail
 # Run the test
 rm -f smoke.serial

-export QEMU_CMD="qemu-system-riscv64 \
+export TEST_CMD="qemu-system-riscv64 \
     -M virt \
     -smp 1 \
     -nographic \
     -m 2g \
     -kernel binaries/xen"

-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
 export PASSED="All set up"

-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-smoke-x86-64.sh 
b/automation/scripts/qemu-smoke-x86-64.sh
index 7495185d9fc6..da0c26cc2f82 100755
--- a/automation/scripts/qemu-smoke-x86-64.sh
+++ b/automation/scripts/qemu-smoke-x86-64.sh
@@ -15,12 +15,12 @@ case $variant in
 esac

 rm -f smoke.serial
-export QEMU_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \
+export TEST_CMD="qemu-system-x86_64 -nographic -kernel binaries/xen \
         -initrd xtf/tests/example/$k \
         -append \"loglvl=all console=com1 noreboot console_timestamps=boot 
$extra\" \
         -m 512 -monitor none -serial stdio"

-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
 export PASSED="Test result: SUCCESS"

-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh 
b/automation/scripts/qemu-xtf-dom0less-arm64.sh
index acef1637e25b..9608de6ec033 100755
--- a/automation/scripts/qemu-xtf-dom0less-arm64.sh
+++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh
@@ -50,7 +50,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d 
binaries/ -c binaries/conf

 # Run the test
 rm -f smoke.serial
-export QEMU_CMD="./binaries/qemu-system-aarch64 \
+export TEST_CMD="./binaries/qemu-system-aarch64 \
     -machine virtualization=true \
     -cpu cortex-a57 -machine type=virt \
     -m 2048 -monitor none -serial stdio \
@@ -61,7 +61,7 @@ export QEMU_CMD="./binaries/qemu-system-aarch64 \
     -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin"

 export UBOOT_CMD="virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 
0x40000000"
-export QEMU_LOG="smoke.serial"
+export TEST_LOG="smoke.serial"
 export PASSED="${passed}"

-./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
+./automation/scripts/console.exp | sed 's/\r\+$//'
diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh 
b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
index 4559e2b9ee1f..ef6e1361a95c 100755
--- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
+++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
@@ -1,8 +1,8 @@
-#!/bin/sh
+#!/usr/bin/env bash

 # Run x86_64 dom0 tests on hardware.

-set -ex
+set -ex -o pipefail

 fatal() {
     echo "$(basename "$0") $*" >&2
@@ -27,7 +27,6 @@ memory = 512
 vif = [ "bridge=xenbr0", ]
 disk = [ ]
 '
-TIMEOUT_SECONDS=200

 # Select test variant.
 if [ "${TEST}" = "ping" ]; then
@@ -125,20 +124,19 @@ boot

 # Power cycle board and collect serial port output.
 SERIAL_DEV="/dev/serial/${TEST_BOARD}"
-SERIAL_CMD="cat ${SERIAL_DEV} | tee smoke.serial | sed 's/\r//'"
 sh /scratch/gitlab-runner/${TEST_BOARD}.sh 2
 sleep 5
 sh /scratch/gitlab-runner/${TEST_BOARD}.sh 1
 sleep 5
 set +e
 stty -F ${SERIAL_DEV} 115200
-timeout -k 1 ${TIMEOUT_SECONDS} nohup sh -c "${SERIAL_CMD}"
-sh /scratch/gitlab-runner/${TEST_BOARD}.sh 2
-
-set -e

-if grep -q "${PASS_MSG}" smoke.serial; then
-    exit 0
-fi
+# Capture test result and power off board before exiting.
+export PASSED="${PASS_MSG}"
+export TEST_CMD="cat ${SERIAL_DEV}"
+export TEST_LOG="smoke.serial"

-fatal "Test failed"
+./automation/scripts/console.exp | sed 's/\r\+$//'
+TEST_RESULT=$?
+sh "/scratch/gitlab-runner/${TEST_BOARD}.sh" 2
+exit ${TEST_RESULT}
diff --git a/automation/scripts/xilinx-smoke-dom0less-arm64.sh 
b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
index 18aa07f0a273..b24ad11b8cac 100755
--- a/automation/scripts/xilinx-smoke-dom0less-arm64.sh
+++ b/automation/scripts/xilinx-smoke-dom0less-arm64.sh
@@ -1,6 +1,6 @@
 #!/bin/bash

-set -ex
+set -ex -o pipefail

 test_variant=$1

@@ -137,13 +137,14 @@ cd $START
 SERIAL_DEV="/dev/serial/zynq"
 set +e
 stty -F ${SERIAL_DEV} 115200
-timeout -k 1 120 nohup sh -c "cat ${SERIAL_DEV} | tee smoke.serial | sed 
's/\r//'"

-# stop the board
-cd /scratch/gitlab-runner
-bash zcu102.sh 2
-cd $START
+# Capture test result and power off board before exiting.
+export PASSED="${passed}"
+export LOG_MSG="Welcome to Alpine Linux"
+export TEST_CMD="cat ${SERIAL_DEV}"
+export TEST_LOG="smoke.serial"

-set -e
-(grep -q "^Welcome to Alpine Linux" smoke.serial && grep -q "${passed}" 
smoke.serial) || exit 1
-exit 0
+./automation/scripts/console.exp | sed 's/\r\+$//'
+TEST_RESULT=$?
+sh "/scratch/gitlab-runner/zcu102.sh" 2
+exit ${TEST_RESULT}
--
2.25.1




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.