Re: [PATCH v2 6/6] CI: run the riscv64 smoke test via QTB framework console-test

["Alejandro Vallejo" <alejandro.garciavallejo@xxxxxxx>]

On Thu Aug 27, 2026 at 11:42 AM CEST, Baptiste Le Duc wrote:
> qemu-smoke-riscv64-gcc drove QEMU through
> automation/scripts/qemu-smoke-riscv64.sh, an expect wrapper whose machine
> description (cpus, memory, device tree, console wiring) lived in the script
> itself. The QTB framework now owns all of that: machines come from the
> shared catalog, expectations from the test type's own YAML.
>
> Turn .qemu-riscv64 into a template running qemu_smoke_riscv64.py <type> run
> <test> in the qtb-riscv64 container, machine and test picked per job
> through QTB_TEST_TYPE/QTB_TEST. The container comes from the test-artifacts
> registry, hence the new ARTIFACTS_REGISTRY next to the existing
> ARTIFACTS_REPO/ARTIFACTS_BRANCH. QTB_BINARIES_DIR points at the artifacts
> of the job (Xen only currently, but aims to have initrd and linux images
> when dom0less will be supported). QTB_LOG_DIR collects the per-console
> logs, kept on failure and on success.
>
> Point qemu-smoke-riscv64-gcc at that template, running the console-test
> type on dom0less-1smp-0domu-1vcpu-aplic-imsic-null: a Xen-only machine, so
> the smoke check is Xen's own "All set up" on console 0, the same string the
> expect script waited for.
>
> Drop automation/scripts/qemu-smoke-riscv64.sh as it has no caller left in
> the CI after this patch and drop smoke.serial from the .qemu-riscv64
> artifacts since no riscv64 job uses it anymore, the logs are now kept under
> QTB_LOG_DIR.
>
> Assisted-by: Claude:claude-opus-5
> Signed-off-by: Baptiste Le Duc <baptiste.le-duc@xxxxxxxxxx>

This is all quite large and monolithic to review quickly, but I can
already tell you that replacing a <50LoC test with such a massive
infra bench is probably not quite what you want at this point in time.

IMO, you ought to keep the existing test in place and perhaps integrate
the new infra with new tests that would be complicated without it.
Things like injecting bizarre interrupts (NMI?) and capture expected
behaviour.

Or even duplicating the existing test would be fine. Build confidence in
it and only after you really trust it do remove your previous smoke.

My .05 cents at least.

Cheers,
Alejandro

> ---
>  .gitlab-ci.yml                           |  3 +++
>  automation/gitlab-ci/test.yaml           | 20 ++++++++++++++------
>  automation/scripts/qemu-smoke-riscv64.sh | 19 -------------------
>  3 files changed, 17 insertions(+), 25 deletions(-)
>  delete mode 100755 automation/scripts/qemu-smoke-riscv64.sh
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index f42a9abeaa..15f93b8634 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -11,6 +11,9 @@ variables:
>    ARTIFACTS_BRANCH:
>      description: "Branch in test-artifacts to use"
>      value: master
> +  ARTIFACTS_REGISTRY:
> +    description: "Registry holding the test-artifacts containers"
> +    value: registry.gitlab.com/xen-project/hardware/test-artifacts
>    LINUX_JOB_X86_64:
>      description: "Job name in test-artifacts to use for Linux x86_64"
>      value: linux-6.6.56-x86_64
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 61adc1baff..e9dd147380 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -72,14 +72,21 @@
>      TEST_TIMEOUT_OVERRIDE: 120
>  
>  .qemu-riscv64:
> +  image: ${ARTIFACTS_REGISTRY}/${CONTAINER}
>    extends: .test-jobs-common
>    variables:
> -    CONTAINER: debian:13-riscv64
> -    LOGFILE: qemu-smoke-riscv64.log
> +    CONTAINER: debian:13-qtb-riscv64
> +    QTB_LOG_DIR: qtb-logs
> +    QTB_BINARIES_DIR: ${CI_PROJECT_DIR}/binaries
> +  script:
> +    - ./automation/scripts/qemu_smoke_riscv64.py
> +      ${QTB_TEST_TYPE}
> +      run
> +      ${QTB_TEST}
> +      --log-dir ${QTB_LOG_DIR}
>    artifacts:
>      paths:
> -      - smoke.serial
> -      - '*.log'
> +      - ${QTB_LOG_DIR}
>      when: always
>    tags:
>      - x86_64
> @@ -779,8 +786,9 @@ qemu-xtf-argo-x86_64-gcc-debug:
>  
>  qemu-smoke-riscv64-gcc:
>    extends: .qemu-riscv64
> -  script:
> -    - ./automation/scripts/qemu-smoke-riscv64.sh 2>&1 | tee ${LOGFILE}
> +  variables:
> +    QTB_TEST_TYPE: console-test
> +    QTB_TEST: dom0less-1smp-0domu-1vcpu-aplic-imsic-null
>    needs:
>      - debian-13-riscv64-gcc-debug
>  
> diff --git a/automation/scripts/qemu-smoke-riscv64.sh 
> b/automation/scripts/qemu-smoke-riscv64.sh
> deleted file mode 100755
> index c0b1082a08..0000000000
> --- a/automation/scripts/qemu-smoke-riscv64.sh
> +++ /dev/null
> @@ -1,19 +0,0 @@
> -#!/bin/bash
> -
> -set -ex -o pipefail
> -
> -# Run the test
> -rm -f smoke.serial
> -
> -export TEST_CMD="qemu-system-riscv64 \
> -    -M virt,aia=aplic-imsic \
> -    -cpu rv64,svpbmt=on \
> -    -smp 1 \
> -    -nographic \
> -    -m 2g \
> -    -kernel binaries/xen"
> -
> -export TEST_LOG="smoke.serial"
> -export PASSED="All set up"
> -
> -./automation/scripts/console.exp |& sed 's/\r\+$//'