Re: [PATCH 2/4] Overhaul how Argo is built and packged

[Denis Mukhin <dmkhn@xxxxxxxxx>]

On Thursday, April 10th, 2025 at 2:37 PM, Andrew Cooper 
<andrew.cooper3@xxxxxxxxxx> wrote:

> 
> 
> Right now, the argo artefacts are a pile of files which the test has to turn
> back into something which resembles a filesystem. Furthermore, because we do
> not build modules for the main kernel, it is extra important to make sure that
> xen-argo.ko doesn't get out of sync.
> 
> Build argo unconditionally as part of the linux artefact. It's ~100kb all
> together, compared to ~14M for the kernel.
> 
> Produce a single argo.cpio.gz with xen-argo.ko in the standard location.
> Prune userspace down to just the executables and libraries.
> 
> This is cribbed from the existing scripts/x86_64-linux-argo.sh, which stays in
> place in the short term until Xen can be updated to use the new scheme.
> 
> Signed-off-by: Andrew Cooper andrew.cooper3@xxxxxxxxxx
> 
> ---
> CC: Anthony PERARD anthony.perard@xxxxxxxxxx
> 
> CC: Stefano Stabellini sstabellini@xxxxxxxxxx
> 
> CC: Michal Orzel michal.orzel@xxxxxxx
> 
> CC: Doug Goldstein cardoe@xxxxxxxxxx
> 
> CC: Marek Marczykowski-Górecki marmarek@xxxxxxxxxxxxxxxxxxxxxx
> 
> CC: Jason Andryuk jason.andryuk@xxxxxxx
> 
> CC: Daniel P. Smith dpsmith@xxxxxxxxxxxxxxxxxxxx
> 
> 
> I tried to make MODPOST work properly, but we don't build enough of it for the
> kernel, and I didn't feel like adding an extra 10 mins to the build (all
> modules) just to get the metadata right.
> ---
> .gitlab-ci.yml | 2 ++
> scripts/build-argo.sh | 67 ++++++++++++++++++++++++++++++++++++++++++
> scripts/build-linux.sh | 6 +++-
> 3 files changed, 74 insertions(+), 1 deletion(-)
> create mode 100644 scripts/build-argo.sh
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index fb997cc62162..790a6d9f9896 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -47,6 +47,8 @@ linux-6.6.56-x86_64:
> script: ./scripts/build-linux.sh
> variables:
> LINUX_VERSION: 6.6.56
> + ARGO_SHA: "705a7a8a624b42e13e655d3042059b8a85cdf6a3"
> + ARGOEXEC_SHA: "d900429f6640acc6f68a3d3a4c945d7da60625d8"
> 
> #
> # The jobs below here are legacy and being phased out.
> diff --git a/scripts/build-argo.sh b/scripts/build-argo.sh
> new file mode 100644
> index 000000000000..0b4005f3b9a0
> --- /dev/null
> +++ b/scripts/build-argo.sh
> @@ -0,0 +1,67 @@
> +#
> +# This is a partial script, sourced by build-linux.sh
> +# It has expectations about the environment
> +#
> +
> +cd "${WORKDIR}"
> +
> +#
> +# We're going to collect everything in argo.cpio.gz. Construct it under
> +# $ARGODIR as we go.
> +#
> +ARGODIR="${WORKDIR}/argo-root"
> +
> +git clone https://github.com/OpenXT/linux-xen-argo.git --depth=1
> +git -C "${WORKDIR}/linux-xen-argo" fetch origin "${ARGO_SHA}"
> +git -C "${WORKDIR}/linux-xen-argo" switch --detach FETCH_HEAD
> +
> +# Build xen-argo.ko against the target kernel, and install it. Install
> +# linux/argo.h too because userspace needs it.
> +make -C "linux-${LINUX_VERSION}" \
> + M="${WORKDIR}/linux-xen-argo/argo-linux" \
> + KBUILD_MODPOST_WARN=1 \
> + CFLAGS_MODULE="-Wno-error" \

Suggest adding `W=e` for catching problems and `V=1` to help debugging a bit.

> + modules
> +install -D -m644 "${WORKDIR}/linux-xen-argo/argo-linux/xen-argo.ko" \
> + "${ARGODIR}/lib/modules/${LINUX_VERSION}/updates/xen-argo.ko"
> +install -D -m644 "${WORKDIR}/linux-xen-argo/argo-linux/include/linux/argo.h" 
> \
> + "${ARGODIR}/usr/include/linux/argo.h"
> +
> +# Build and install libargo, applying fixes to build in Alpine Linux
> +cd "${WORKDIR}/linux-xen-argo/libargo"
> +sed -e "s|AM_INIT_AUTOMAKE|AC_CONFIG_AUX_DIR(.)\nAM_INIT_AUTOMAKE|" \
> + -i configure.ac
> +sed -e "s/_SOCKADDR_COMMON (sxenargo)/sa_family_t sxenargo_family/" \
> + -e "s/__SOCKADDR_COMMON_SIZE/(sizeof (unsigned short int))/" \
> + -i src/libargo.h
> +
> +autoreconf --install
> +./configure --prefix=/usr CPPFLAGS="-I${PWD}/../argo-linux/include"
> +make

Perhaps add something like `make -j$(nproc)`?

> +make install DESTDIR="${ARGODIR}"
> +
> +# Build and install argo-exec, modifying for xilinx argo test
> +cd "${WORKDIR}"
> +curl -fsSLO \
> + 
> https://raw.githubusercontent.com/OpenXT/xenclient-oe/${ARGOEXEC_SHA}/recipes-openxt/argo-exec/argo-exec/argo-exec.c
> +sed -e "/#include <xen\/xen.h>/d" \
> 
> + -e "s|ret = shuffle(s, fds\[0\], fds\[1\]);|ret = shuffle(s, 0, 1);|" \
> + -i argo-exec.c
> +
> +gcc -I"${ARGODIR}/usr/include" -L"${ARGODIR}/usr/lib/" \
> + argo-exec.c -o "${ARGODIR}/usr/bin/argo-exec" -largo
> +
> +#
> +# Building is now complete. Strip the devel components and the nointerposer
> +# lib, which we don't care to deploy to the test system.
> +#
> +cd $ARGODIR
> +rm -r usr/include usr/lib/pkgconfig
> +find usr/lib -name \nointerposer\ -delete
> +find usr/lib \( -name \.a -o -name \.so -o -name \*.la \) -delete
> +
> +# Package everything up
> +find . | cpio -R 0:0 -H newc -o | gzip > "$COPYDIR/argo.cpio.gz"
> 
> +
> +# Print the contents for the build log
> +zcat "$COPYDIR/argo.cpio.gz" | cpio -tv
> diff --git a/scripts/build-linux.sh b/scripts/build-linux.sh
> index 652fdba7b9d1..49b5ebed6424 100755
> --- a/scripts/build-linux.sh
> +++ b/scripts/build-linux.sh
> @@ -8,7 +8,7 @@ fi
> set -ex -o pipefail
> 
> WORKDIR="${PWD}"
> -COPYDIR="${WORKDIR}/binaries/"
> +COPYDIR="${WORKDIR}/binaries"
> UNAME=$(uname -m)
> 
> # Build Linux
> @@ -45,6 +45,10 @@ case $UNAME in
> x86_64)
> make -j$(nproc) bzImage
> cp arch/x86/boot/bzImage "${COPYDIR}"
> +
> + # Build argo
> + make modules_prepare
> + . "${WORKDIR}/scripts/build-argo.sh"

Nit: I would use `source` instead of `.` which is simple to overlook.

> ;;
> 
> aarch64)
> --
> 2.39.5