[PATCH test-artifacts] Build minimal images with Buildroot

[Titouan Christophe <titouan.christophe@xxxxxxx>]

Add a way to build minimal Xen + Linux dom0 images, and their corresponding
cross-compilation toolchains with Buildroot [1]. This effectively provides
a consistent cross-architecture development environment.

Buildroot is a project that makes it easy to build embedded system images,
including Linux, Xen or bootloaders. Leverage that build system to build
minimal images for use in the CI environment of Xen sub-projects, namely
Unikraft.

This approach is based on a Buildroot External tree [2] which currently only
provides 2 target defconfigs (arm64 and x86_64, both running on qemu).
It uses a recent development version of Buildroot that integrates changes
to build Xen on x86_64 [3], on its latest official version 4.22.0 [4].
It also includes the latest kernel (linux-7.2.7 at the time of writing).

As a byproduct, aside xen+kernel+rootfs, we also get a cross-compilation
toolchain (that runs on x86_64 hosts). This makes it easy to cross-compile
domUs for arm64, and it alleviates dynamic libraries mismatches when
adding userspace programs to the dom0 Linux image. As an example, one such
problem: Alpine linux is based on musl while your regular development
computer is probably based on glibc, which causes issues if you are
recompiling `xl` and try to use it in the Alpine image.

[1] https://buildroot.org/
[2] https://buildroot.org/downloads/manual/manual.html#outside-br-custom
[3] 
https://gitlab.com/buildroot.org/buildroot/-/commit/0bf8b2ecee6b1b41746e8863b57c2c5ba3a623b1
[4] 
https://gitlab.com/buildroot.org/buildroot/-/commit/a962be2eea146e06661712159698bc7cbf3305f2

This work is derived from this original series:
https://gitlab.com/xen-project/fusa/unikraft/-/merge_requests/1

Signed-off-by: Titouan Christophe <titouan.christophe@xxxxxxx>
---
 .gitlab-ci.yml                                | 20 ++++++++++
 br2-xendevel-external/.gitignore              |  2 +
 br2-xendevel-external/Config.in               |  9 +++++
 .../board/xendevel/common/rootfs-prune.sh     | 21 ++++++++++
 .../xendevel/x86_64/linux-xen-dom0.fragment   | 40 +++++++++++++++++++
 .../configs/xendevel_dom0_arm64_defconfig     | 34 ++++++++++++++++
 .../configs/xendevel_dom0_x86_64_defconfig    | 38 ++++++++++++++++++
 br2-xendevel-external/external.desc           |  2 +
 br2-xendevel-external/external.mk             |  9 +++++
 scripts/build-buildroot.sh                    | 35 ++++++++++++++++
 10 files changed, 210 insertions(+)
 create mode 100644 br2-xendevel-external/.gitignore
 create mode 100644 br2-xendevel-external/Config.in
 create mode 100755 br2-xendevel-external/board/xendevel/common/rootfs-prune.sh
 create mode 100644 
br2-xendevel-external/board/xendevel/x86_64/linux-xen-dom0.fragment
 create mode 100644 br2-xendevel-external/configs/xendevel_dom0_arm64_defconfig
 create mode 100644 br2-xendevel-external/configs/xendevel_dom0_x86_64_defconfig
 create mode 100644 br2-xendevel-external/external.desc
 create mode 100644 br2-xendevel-external/external.mk
 create mode 100755 scripts/build-buildroot.sh

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index d04822f..8db3a0b 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -81,3 +81,23 @@ linux-6.6.56-x86_64:
 microcode-x86:
   extends: .x86_64-artifacts
   script: ./scripts/x86-microcode.sh
+
+
+#
+# Buildroot based artifacts
+#
+buildroot:
+  extends: .artifacts
+  image:
+    name: registry.gitlab.com/buildroot.org/buildroot/base:20250218.2110
+  parallel:
+    matrix:
+      - VARIANT: [arm64, x86_64]
+  cache:
+    key: buildroot-dl-${VARIANT}
+    paths:
+      - br2-xendevel-external/dl/
+  script:
+    - ./scripts/build-buildroot.sh "${VARIANT}"
+    - mkdir -p binaries
+    - cp br2-xendevel-external/xen-${VARIANT}/images/* binaries/
diff --git a/br2-xendevel-external/.gitignore b/br2-xendevel-external/.gitignore
new file mode 100644
index 0000000..424351b
--- /dev/null
+++ b/br2-xendevel-external/.gitignore
@@ -0,0 +1,2 @@
+xen-arm64
+xen-x86_64
diff --git a/br2-xendevel-external/Config.in b/br2-xendevel-external/Config.in
new file mode 100644
index 0000000..3c8e444
--- /dev/null
+++ b/br2-xendevel-external/Config.in
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+#
+# Kconfig entry point for the XENDEVEL external tree. Entries appear under
+# "External options -> Xen dom0 + Unikraft domU development environment" in
+# `make menuconfig`.
+#
+# No XENDEVEL packages are defined yet; add
+#   source "$BR2_EXTERNAL_XENDEVEL_PATH/package/<name>/Config.in"
+# lines here as they are introduced.
diff --git a/br2-xendevel-external/board/xendevel/common/rootfs-prune.sh 
b/br2-xendevel-external/board/xendevel/common/rootfs-prune.sh
new file mode 100755
index 0000000..7e3a661
--- /dev/null
+++ b/br2-xendevel-external/board/xendevel/common/rootfs-prune.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+# SPDX-License-Identifier: BSD-3-Clause
+#
+# Buildroot post-build script: shrink the dom0 target filesystem so the cpio
+# initramfs unpacks inside the dom0 memory window. $1 is the target directory.
+set -e
+T="$1"
+
+# qemu-xen BIOS/ROM/keymap blobs (~283MB) — unused by PV/PVH guests.
+rm -rf "$T/usr/share/qemu-xen"
+
+# Docs / man / info — not needed at runtime.
+rm -rf "$T/usr/share/doc" "$T/usr/share/man" "$T/usr/share/info"
+
+# The arch default kernel config is monolithic enough to boot dom0 without
+# loadable modules for the smoke test. Keep the xen-related modules if present,
+# drop the rest to shrink the initramfs.
+if [ -d "$T/lib/modules" ]; then
+       find "$T/lib/modules" -type f -name '*.ko' \
+               ! -path '*xen*' -delete 2>/dev/null || true
+fi
diff --git 
a/br2-xendevel-external/board/xendevel/x86_64/linux-xen-dom0.fragment 
b/br2-xendevel-external/board/xendevel/x86_64/linux-xen-dom0.fragment
new file mode 100644
index 0000000..a999333
--- /dev/null
+++ b/br2-xendevel-external/board/xendevel/x86_64/linux-xen-dom0.fragment
@@ -0,0 +1,40 @@
+# Linux kernel config fragment: x86_64 Xen dom0.
+#
+# The stock x86_64 kernel defconfig does NOT enable Xen (CONFIG_XEN is not 
set),
+# so the resulting bzImage carries no Xen PV ELF notes / PVH entry and Xen
+# refuses to build dom0 ("Will only load images built for the generic loader or
+# Linux images ... or with PHYS32_ENTRY set" / "Could not construct d0"). Turn 
on
+# Xen PV+PVH dom0, the hvc console, and the backend/toolstack devices (evtchn,
+# gntdev, gntalloc, xenfs, blk/net back) so the xl toolstack can drive the 
domU.
+CONFIG_HYPERVISOR_GUEST=y
+CONFIG_PARAVIRT=y
+CONFIG_XEN=y
+CONFIG_XEN_PV=y
+CONFIG_XEN_PV_SMP=y
+CONFIG_XEN_DOM0=y
+CONFIG_XEN_PV_DOM0=y
+CONFIG_XEN_PVH=y
+CONFIG_XEN_PVHVM=y
+CONFIG_XEN_SAVE_RESTORE=y
+CONFIG_XEN_BACKEND=y
+CONFIG_XENFS=y
+CONFIG_XEN_COMPAT_XENFS=y
+CONFIG_XEN_SYS_HYPERVISOR=y
+CONFIG_XEN_XENBUS_FRONTEND=y
+CONFIG_XEN_GNTDEV=y
+CONFIG_XEN_GRANT_DEV_ALLOC=y
+CONFIG_XEN_DEV_EVTCHN=y
+CONFIG_XEN_PRIVCMD=y
+CONFIG_XEN_BLKDEV_FRONTEND=y
+CONFIG_XEN_BLKDEV_BACKEND=y
+CONFIG_XEN_NETDEV_FRONTEND=y
+CONFIG_XEN_NETDEV_BACKEND=y
+CONFIG_HVC_XEN=y
+CONFIG_HVC_XEN_FRONTEND=y
+CONFIG_TTY=y
+# Monolithic kernel: build everything in, no loadable modules. This drops the
+# whole udev/libkmod module-loading path at boot (no .ko files to insert), so
+# the rootfs can also drop kmod + eudev. Every driver this QEMU dom0 needs
+# (ata_piix, e1000, virtio, 9p, xen blk/net back) is already =y in the x86_64
+# arch default config, so nothing required is lost.
+# CONFIG_MODULES is not set
diff --git a/br2-xendevel-external/configs/xendevel_dom0_arm64_defconfig 
b/br2-xendevel-external/configs/xendevel_dom0_arm64_defconfig
new file mode 100644
index 0000000..cb802a8
--- /dev/null
+++ b/br2-xendevel-external/configs/xendevel_dom0_arm64_defconfig
@@ -0,0 +1,34 @@
+# aarch64 Xen dom0: hypervisor + xl toolstack + Linux
+
+BR2_aarch64=y
+BR2_cortex_a72=y
+
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
+BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_AARCH64_GLIBC_STABLE=y
+
+BR2_TARGET_GENERIC_HOSTNAME="xen-dom0"
+BR2_TARGET_GENERIC_ISSUE="Xen dom0 (br2-xendevel-external)"
+BR2_TARGET_GENERIC_GETTY_PORT="console"
+
+# devtmpfs mounted by init + eudev, so /dev/null and the Xen
+# gnttab/evtchn/privcmd device nodes exist at runtime.
+BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV=y
+
+BR2_LINUX_KERNEL=y
+BR2_LINUX_KERNEL_USE_ARCH_DEFAULT_CONFIG=y
+BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
+
+BR2_PACKAGE_XEN=y
+BR2_PACKAGE_XEN_HYPERVISOR=y
+BR2_PACKAGE_XEN_TOOLS=y
+
+BR2_PACKAGE_KMOD=y
+BR2_PACKAGE_UTIL_LINUX_BINARIES=y
+
+BR2_TARGET_ROOTFS_CPIO=y
+BR2_TARGET_ROOTFS_CPIO_GZIP=y
+# BR2_TARGET_ROOTFS_TAR is not set
+BR2_ROOTFS_POST_BUILD_SCRIPT="$(BR2_EXTERNAL_XENDEVEL_PATH)/board/xendevel/common/rootfs-prune.sh"
+
+BR2_PACKAGE_HOST_ENVIRONMENT_SETUP=y
diff --git a/br2-xendevel-external/configs/xendevel_dom0_x86_64_defconfig 
b/br2-xendevel-external/configs/xendevel_dom0_x86_64_defconfig
new file mode 100644
index 0000000..7e0d894
--- /dev/null
+++ b/br2-xendevel-external/configs/xendevel_dom0_x86_64_defconfig
@@ -0,0 +1,38 @@
+# x86_64 Xen dom0: hypervisor + xl toolstack + Linux
+
+BR2_x86_64=y
+
+BR2_TOOLCHAIN_EXTERNAL=y
+BR2_TOOLCHAIN_EXTERNAL_BOOTLIN=y
+BR2_TOOLCHAIN_EXTERNAL_BOOTLIN_X86_64_GLIBC_STABLE=y
+
+BR2_TARGET_GENERIC_HOSTNAME="xen-dom0"
+BR2_TARGET_GENERIC_ISSUE="Xen dom0 (br2-xendevel-external)"
+BR2_TARGET_GENERIC_GETTY_PORT="console"
+
+# The monolithic kernel has CONFIG_DEVTMPFS_MOUNT=y, so it creates every dev
+# (including the Xen control nodes /dev/xen/{evtchn,gntdev,privcmd,...} that
+# xenstored/xenconsoled need) at driver registration. That removes the need for
+# a userspace device manager and the libkmod/udev boot noise entirely.
+BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_DEVTMPFS=y
+
+BR2_LINUX_KERNEL=y
+BR2_LINUX_KERNEL_USE_ARCH_DEFAULT_CONFIG=y
+BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES="$(BR2_EXTERNAL_XENDEVEL_PATH)/board/xendevel/x86_64/linux-xen-dom0.fragment"
+BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
+BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF=y
+
+BR2_PACKAGE_XEN=y
+BR2_PACKAGE_XEN_HYPERVISOR=y
+BR2_PACKAGE_XEN_TOOLS=y
+
+# No kmod: the kernel is monolithic (CONFIG_MODULES=n), so there are no modules
+# to insert and nothing links against libkmod. Dropping it (and eudev, above)
+# removes the only consumer that was failing on the missing libkmod.so.2.
+
+BR2_TARGET_ROOTFS_CPIO=y
+BR2_TARGET_ROOTFS_CPIO_GZIP=y
+# BR2_TARGET_ROOTFS_TAR is not set
+BR2_ROOTFS_POST_BUILD_SCRIPT="$(BR2_EXTERNAL_XENDEVEL_PATH)/board/xendevel/common/rootfs-prune.sh"
+
+BR2_PACKAGE_HOST_ENVIRONMENT_SETUP=y
diff --git a/br2-xendevel-external/external.desc 
b/br2-xendevel-external/external.desc
new file mode 100644
index 0000000..232eb0e
--- /dev/null
+++ b/br2-xendevel-external/external.desc
@@ -0,0 +1,2 @@
+name: XENDEVEL
+desc: Xen development environment
diff --git a/br2-xendevel-external/external.mk 
b/br2-xendevel-external/external.mk
new file mode 100644
index 0000000..37cd1d8
--- /dev/null
+++ b/br2-xendevel-external/external.mk
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: BSD-3-Clause
+#
+# Buildroot external tree makefile fragment.
+#
+# Every package/*/*.mk under this tree is included by Buildroot. There are no
+# XENDEVEL-specific packages yet: the tree currently provides the dom0
+# defconfigs and board support files only.
+
+include $(sort $(wildcard $(BR2_EXTERNAL_XENDEVEL_PATH)/package/*/*.mk))
diff --git a/scripts/build-buildroot.sh b/scripts/build-buildroot.sh
new file mode 100755
index 0000000..73ab123
--- /dev/null
+++ b/scripts/build-buildroot.sh
@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+
+set -e -o pipefail
+
+BR2_EXT_DIR=$(realpath $(dirname $0)/../br2-xendevel-external)
+
+BR_REPO=https://gitlab.com/buildroot.org/buildroot
+# First Buildroot version (2026.08-345-ga962be2eea) with Xen 4.22 on x86_64
+BR_COMMIT=a962be2eea146e06661712159698bc7cbf3305f2
+BR_DIR=${BR2_EXT_DIR}/buildroot
+export BR2_DL_DIR=${BR2_EXT_DIR}/dl
+
+# Not using `git clone --revision` as it requires git >= 2.49
+if [ "$(git -C ${BR_DIR} rev-parse HEAD 2>/dev/null)" != "${BR_COMMIT}" ]; then
+       git init -q ${BR_DIR}
+       git -C ${BR_DIR} fetch --depth=1 ${BR_REPO} ${BR_COMMIT}
+       git -C ${BR_DIR} checkout -q FETCH_HEAD
+fi
+
+do_build() {   
+       build_dir=${BR2_EXT_DIR}/xen-${1}
+       make BR2_EXTERNAL=${BR2_EXT_DIR} O=${build_dir} -C ${BR_DIR} 
xendevel_dom0_${1}_defconfig
+       make -C ${build_dir} source all sdk | grep '>>>'
+}
+
+for arg in $*; do
+       case "$arg" in
+               arm64|x86_64)
+                       do_build $arg
+                       ;;
+               *)
+                       echo "Unknown target $arg" >&2
+                       exit 1
+       esac
+done
-- 
2.55.0