Re: [PATCH v2 6/6] CI: run the riscv64 smoke test via QTB framework console-test

[Baptiste Le Duc <baptiste.le-duc@xxxxxxxxxx>]

On 2026-09-21 16:37 +0200, Alejandro Vallejo wrote:
> On Thu Aug 27, 2026 at 11:42 AM CEST, Baptiste Le Duc wrote:
> > qemu-smoke-riscv64-gcc drove QEMU through
> > automation/scripts/qemu-smoke-riscv64.sh, an expect wrapper whose machine
> > description (cpus, memory, device tree, console wiring) lived in the script
> > itself. The QTB framework now owns all of that: machines come from the
> > shared catalog, expectations from the test type's own YAML.
> >
> > Turn .qemu-riscv64 into a template running qemu_smoke_riscv64.py <type> run
> > <test> in the qtb-riscv64 container, machine and test picked per job
> > through QTB_TEST_TYPE/QTB_TEST. The container comes from the test-artifacts
> > registry, hence the new ARTIFACTS_REGISTRY next to the existing
> > ARTIFACTS_REPO/ARTIFACTS_BRANCH. QTB_BINARIES_DIR points at the artifacts
> > of the job (Xen only currently, but aims to have initrd and linux images
> > when dom0less will be supported). QTB_LOG_DIR collects the per-console
> > logs, kept on failure and on success.
> >
> > Point qemu-smoke-riscv64-gcc at that template, running the console-test
> > type on dom0less-1smp-0domu-1vcpu-aplic-imsic-null: a Xen-only machine, so
> > the smoke check is Xen's own "All set up" on console 0, the same string the
> > expect script waited for.
> >
> > Drop automation/scripts/qemu-smoke-riscv64.sh as it has no caller left in
> > the CI after this patch and drop smoke.serial from the .qemu-riscv64
> > artifacts since no riscv64 job uses it anymore, the logs are now kept under
> > QTB_LOG_DIR.
> >
> > Assisted-by: Claude:claude-opus-5
> > Signed-off-by: Baptiste Le Duc <baptiste.le-duc@xxxxxxxxxx>
> 
Hi Alejandro,
> This is all quite large and monolithic to review quickly, but I can
> already tell you that replacing a <50LoC test with such a massive
> infra bench is probably not quite what you want at this point in time.
Thanks a lot for this quick review. Sorry for these long patches, I will
split them to make the review easier.
> 
> IMO, you ought to keep the existing test in place and perhaps integrate
> the new infra with new tests that would be complicated without it.
> Things like injecting bizarre interrupts (NMI?) and capture expected
> behaviour.
> 
> Or even duplicating the existing test would be fine. Build confidence in
> it and only after you really trust it do remove your previous smoke.
Oh I don't know we could proceed this way... Duplicate seems then to be
a good idea. The aim of this patch series was above all to introduce the
the core framework which will be used to inject interrupts.
console_test.py replaces the old smoke test but adds per-domU console
testing, which goes beyond the original scope. I'll trim it to match.
> My .05 cents at least.
Thanks again!
> Cheers,
> Alejandro
> 
> > ---
> >  .gitlab-ci.yml                           |  3 +++
> >  automation/gitlab-ci/test.yaml           | 20 ++++++++++++++------
> >  automation/scripts/qemu-smoke-riscv64.sh | 19 -------------------
> >  3 files changed, 17 insertions(+), 25 deletions(-)
> >  delete mode 100755 automation/scripts/qemu-smoke-riscv64.sh
> >
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index f42a9abeaa..15f93b8634 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -11,6 +11,9 @@ variables:
> >    ARTIFACTS_BRANCH:
> >      description: "Branch in test-artifacts to use"
> >      value: master
> > +  ARTIFACTS_REGISTRY:
> > +    description: "Registry holding the test-artifacts containers"
> > +    value: registry.gitlab.com/xen-project/hardware/test-artifacts
> >    LINUX_JOB_X86_64:
> >      description: "Job name in test-artifacts to use for Linux x86_64"
> >      value: linux-6.6.56-x86_64
> > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> > index 61adc1baff..e9dd147380 100644
> > --- a/automation/gitlab-ci/test.yaml
> > +++ b/automation/gitlab-ci/test.yaml
> > @@ -72,14 +72,21 @@
> >      TEST_TIMEOUT_OVERRIDE: 120
> >  
> >  .qemu-riscv64:
> > +  image: ${ARTIFACTS_REGISTRY}/${CONTAINER}
> >    extends: .test-jobs-common
> >    variables:
> > -    CONTAINER: debian:13-riscv64
> > -    LOGFILE: qemu-smoke-riscv64.log
> > +    CONTAINER: debian:13-qtb-riscv64
> > +    QTB_LOG_DIR: qtb-logs
> > +    QTB_BINARIES_DIR: ${CI_PROJECT_DIR}/binaries
> > +  script:
> > +    - ./automation/scripts/qemu_smoke_riscv64.py
> > +      ${QTB_TEST_TYPE}
> > +      run
> > +      ${QTB_TEST}
> > +      --log-dir ${QTB_LOG_DIR}
> >    artifacts:
> >      paths:
> > -      - smoke.serial
> > -      - '*.log'
> > +      - ${QTB_LOG_DIR}
> >      when: always
> >    tags:
> >      - x86_64
> > @@ -779,8 +786,9 @@ qemu-xtf-argo-x86_64-gcc-debug:
> >  
> >  qemu-smoke-riscv64-gcc:
> >    extends: .qemu-riscv64
> > -  script:
> > -    - ./automation/scripts/qemu-smoke-riscv64.sh 2>&1 | tee ${LOGFILE}
> > +  variables:
> > +    QTB_TEST_TYPE: console-test
> > +    QTB_TEST: dom0less-1smp-0domu-1vcpu-aplic-imsic-null
> >    needs:
> >      - debian-13-riscv64-gcc-debug
> >  
> > diff --git a/automation/scripts/qemu-smoke-riscv64.sh 
> > b/automation/scripts/qemu-smoke-riscv64.sh
> > deleted file mode 100755
> > index c0b1082a08..0000000000
> > --- a/automation/scripts/qemu-smoke-riscv64.sh
> > +++ /dev/null
> > @@ -1,19 +0,0 @@
> > -#!/bin/bash
> > -
> > -set -ex -o pipefail
> > -
> > -# Run the test
> > -rm -f smoke.serial
> > -
> > -export TEST_CMD="qemu-system-riscv64 \
> > -    -M virt,aia=aplic-imsic \
> > -    -cpu rv64,svpbmt=on \
> > -    -smp 1 \
> > -    -nographic \
> > -    -m 2g \
> > -    -kernel binaries/xen"
> > -
> > -export TEST_LOG="smoke.serial"
> > -export PASSED="All set up"
> > -
> > -./automation/scripts/console.exp |& sed 's/\r\+$//'
> 
> 
>