[PATCH v3 4/6] automation/qtb: add QTB framework README

[Baptiste Le Duc <baptiste.le-duc@xxxxxxxxxx>]

Document the qtb riscv64 test framework in a README.

It covers:
  - the core concepts: a test type is a pytest module under tests/, a test
    is a function parametrizing the `machine` fixture with what it boots
  - the source files layout
  - the prerequisites and how to run the tests with pytest, with logs and
    a JUnit report as CI does
  - how to run them in the CI container
  - how to add a test and how to add a new test type.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Baptiste Le Duc <baptiste.le-duc@xxxxxxxxxx>
---
Changes in v3:
- Rewrite for the pytest-based framework.
---
 automation/scripts/qtb/riscv/README.md | 127 +++++++++++++++++++++++++++++++++
 1 file changed, 127 insertions(+)

diff --git a/automation/scripts/qtb/riscv/README.md 
b/automation/scripts/qtb/riscv/README.md
new file mode 100644
index 0000000000..f074a70c4a
--- /dev/null
+++ b/automation/scripts/qtb/riscv/README.md
@@ -0,0 +1,127 @@
+qtb riscv64 test framework
+==========================
+
+A small pytest framework built on qtb (QEMU Test Bench), a Python package that
+controls a QEMU process over QMP and qtest and gives access to its consoles.
+Each test drives Xen and its guests through these interfaces and turns what
+it observes into a pass/fail verdict.
+
+Prerequisites
+-------------
+
+- `qemu.qtb`: the QEMU Test Bench Python package. It is not upstream yet, so
+  install it with its `[qtb]` extra (which pulls `pexpect`) from the `python/`
+  directory of the `QEMU_REPO` fork, at the `QEMU_COMMIT` pinned in
+  `automation/build/debian/13-riscv64-qtb.dockerfile`.
+- `pytest`.
+- `qemu-system-riscv64` in PATH, with its bundled OpenSBI.
+- A riscv64 `xen` binary, in the directory given with `--binaries-dir`.
+
+The simplest way to get the Python ones is to use the CI container, see
+"Running in the container" below.
+
+Running the tests
+-----------------
+
+Run from the top level of the source tree, as CI does. Here the binaries are
+in `binaries/`:
+
+```
+T=automation/scripts/qtb/riscv/tests
+
+# List every test:
+python3 -m pytest $T --collect-only -q
+
+# Run every test, one file, one test, or by name substring:
+python3 -m pytest $T --binaries-dir binaries
+python3 -m pytest $T/test_console.py --binaries-dir binaries
+python3 -m pytest $T/test_console.py::test_host_boot --binaries-dir binaries
+python3 -m pytest $T -k 1pcpu --binaries-dir binaries
+
+# With logs and per-test results as JUnit XML (what CI does):
+python3 -m pytest $T --binaries-dir binaries --log-dir qtb-logs \
+    --junitxml tests-junit.xml
+```
+
+pytest finds the `qtb` package on its own, as `automation/scripts/` is the
+first directory above the tests without an `__init__.py`.
+
+The `machine` fixture understands the following options.
+
+- `--binaries-dir DIR`: The directory holding the binaries. It is required to
+  boot a test.
+
+- `--log-dir DIR`: The directory collecting the logs, in one
+  `<module>/<test>/` subdirectory per test. It holds QEMU's stdout/stderr
+  (`riscv64-qtest-<pid>.log`, named after the host) and Xen's console as
+  `con0.log`. Without it, no logs are written.
+
+`--log-cli-level=DEBUG` shows the debug logs live.
+
+Running in the container
+------------------------
+
+Build the CI container once, then run the tests in it with `containerize`,
+which mounts the Xen tree as the working directory. In this example, we assume
+the binaries were copied to `binaries/` at the top of the tree.
+
+```
+make -C automation/build debian/13-riscv64-qtb
+CONTAINER=trixie-riscv64-qtb ./automation/scripts/containerize \
+    python3 -m pytest automation/scripts/qtb/riscv/tests --binaries-dir 
binaries
+```
+
+Layout
+------
+
+```
+qtb/env.py                   QtbEnv: directories handed to each host
+qtb/utils.py                 Generic helpers, not riscv specific
+
+qtb/riscv/                   This framework
+  machine.py                 HostConfig: what a host boots; RiscvTestMachine:
+                             QEMU command line, console helpers
+
+  tests/                     The QEMU tests, one module per test type
+    conftest.py              `machine` fixture and its options
+    test_console.py          console-test
+```
+
+Test types and tests
+--------------------
+
+A test type is one module under `tests/`. Each test is a short function
+taking the `machine` fixture, which it parametrizes indirectly with the
+pCPU counts it runs on:
+
+```
+@pytest.mark.parametrize(
+    "machine",
+    [pytest.param(1, id="1pcpu")],
+    indirect=True,
+)
+def test_a(machine):
+    ...
+```
+
+Before the test body runs, the `machine` fixture launches QEMU with Xen. Once
+the test body returns, QEMU is shut down.
+
+An exception raised before the test body (a bad parameter, a missing binary,
+QEMU not starting) is reported by pytest and JUnit as an *error*. An
+exception raised in the test body (an `expect_console` timing out) is
+reported as a *failure*.
+
+Adding a test
+-------------
+
+Add a function to the module of the test type, parametrize `machine`
+indirectly with the machines it runs on, and check their consoles.
+
+Adding a test type
+------------------
+
+Add `tests/test_<type>.py`, holding the logic of the type in one function
+that takes the machine and the parameters of the type, and the tests that
+call it. Host-side helpers shared between types go to `RiscvTestMachine` or
+`machine.py`. State the requirements of the type in the module docstring.

-- 
2.56.0