Re: [PATCH v4 2/2] gitlab-ci: add an ARM32 qemu-based smoke test

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Wed, 23 Mar 2022, Michal Orzel wrote:
> Hi Stefano,

Hi Michal,

I addressed both points below. Sorry for taking so long but the CI-loop
was blocked on ARM. Now it is running again.



> On 22.03.2022 21:38, Stefano Stabellini wrote:
> > Add a minimal ARM32 smoke test based on qemu-system-arm, as provided by
> > the test-artifacts qemu container. The minimal test simply boots Xen
> > (built from previous build stages) and Dom0.
> > 
> > The test needs a working kernel and minimal initrd for dom0. Instead of
> > building our own kernel and initrd, which would mean maintaining one or
> > two more builting scripts under automation/, we borrow a kernel and
> > initrd from distros.
> > 
> > For the kernel we pick the Debian Bullseye kernel, which has everything
> > we need already built-in. However, we cannot use the Debian Bullseye
> > initrd because it is 22MB and the large size causes QEMU to core dump.
> > 
> > Instead, use the tiny busybox-based rootfs provided by Alpine Linux,
> > which is really minimal: just 2.5MB. Note that we cannot use the Alpine
> > Linux kernel because that doesn't boot on Xen.
> > 
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
> > ---
> > Changes in v4:
> > - improve commit message
> > - use Debian Bullseye kernel
> > - use Alpine Linux initrd
> > ---
> >  automation/gitlab-ci/test.yaml         | 23 ++++++++
> >  automation/scripts/qemu-smoke-arm32.sh | 81 ++++++++++++++++++++++++++
> >  2 files changed, 104 insertions(+)
> >  create mode 100755 automation/scripts/qemu-smoke-arm32.sh
> > 
> > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> > index ec2a2e1607..42cd725a12 100644
> > --- a/automation/gitlab-ci/test.yaml
> > +++ b/automation/gitlab-ci/test.yaml
> > @@ -95,6 +95,29 @@ qemu-smoke-arm64-gcc:
> >      - /^coverity-tested\/.*/
> >      - /^stable-.*/
> >  
> > +qemu-smoke-arm32-gcc:
> > +  stage: test
> > +  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> > +  variables:
> > +    CONTAINER: debian:unstable-arm64v8
> > +  script:
> > +    - ./automation/scripts/qemu-smoke-arm32.sh 2>&1 | tee 
> > qemu-smoke-arm32.log
> > +  dependencies:
> > +    - debian-unstable-gcc-arm32
> > +    - qemu-system-aarch64-6.0.0-arm32-export
> > +  artifacts:
> > +    paths:
> > +      - smoke.serial
> > +      - '*.log'
> > +    when: always
> > +  tags:
> > +    - arm64
> > +  except:
> > +    - master
> > +    - smoke
> > +    - /^coverity-tested\/.*/
> > +    - /^stable-.*/
> > +
> >  qemu-smoke-x86-64-gcc:
> >    stage: test
> >    image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> > diff --git a/automation/scripts/qemu-smoke-arm32.sh 
> > b/automation/scripts/qemu-smoke-arm32.sh
> > new file mode 100755
> > index 0000000000..d554de7939
> > --- /dev/null
> > +++ b/automation/scripts/qemu-smoke-arm32.sh
> > @@ -0,0 +1,81 @@
> > +#!/bin/bash
> > +
> > +set -ex
> > +
> > +export DEBIAN_FRONTENT=noninteractive
> > +apt-get -qy update
> > +apt-get -qy install --no-install-recommends device-tree-compiler \
> > +                                            curl \
> > +                                            cpio
> > +
> > +cd binaries
> > +# Use the kernel from Debian
> > +curl --fail --silent --show-error --location --output vmlinuz 
> > http://http.us.debian.org/debian/dists/bullseye/main/installer-armhf/current/images/netboot/vmlinuz
> > +# Use a tiny initrd based on busybox from Alpine Linux
> > +curl --fail --silent --show-error --location --output initrd.tar.gz 
> > https://dl-cdn.alpinelinux.org/alpine/v3.15/releases/armhf/alpine-minirootfs-3.15.1-armhf.tar.gz
> > +
> > +mkdir rootfs
> > +cd rootfs
> > +tar xvzf ../initrd.tar.gz
> > +find . | cpio -H newc -o | gzip > ../initrd.gz
> > +cd ..
> > +
> > +kernel=`stat -L --printf="%s" vmlinuz`
> > +initrd=`stat -L --printf="%s" initrd.gz`
> > +
> > +# For Xen, we need a couple of more node. Dump the DT from QEMU and add 
> > them
> > +# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
> > +curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
> > +./qemu-system-arm \
> > +   -machine virt-6.0 \
> Can't we just use "virt" as an alias to the latest virt machine available?
> 
> > +   -machine virtualization=true \
> > +   -smp 4 \
> > +   -m 1024 \
> > +   -serial stdio \
> > +   -monitor none \
> > +   -display none \
> > +   -machine dumpdtb=virt.dtb
> > +
> > +dtc -I dtb -O dts virt.dtb > virt.dts
> > +
> > +cat >> virt.dts << EOF
> > +/ {
> > +   chosen {
> > +           #address-cells = <0x2>;
> > +           #size-cells = <0x2>;
> > +           stdout-path = "/pl011@9000000";
> > +        xen,xen-bootargs = "console=dtuart dtuart=/pl011@9000000 
> > dom0_mem=512M bootscrub=0";
> > +           xen,dom0-bootargs = "console=tty0 console=hvc0 earlyprintk 
> > clk_ignore_unused root=/dev/ram0 rdinit=/bin/sh init=/bin/sh";
> As you are using initrd, rdinit is the correct option.
> Specyfing both rdinit and init does not make a lot of sense as the kernel 
> won't reach init= parsing.
> 
> > +           dom0 {
> > +                   compatible = "xen,linux-zimage", "xen,multiboot-module";
> > +                   reg = <0x0 0x1000000 0x0 $kernel>;
> > +           };
> > +        dom0-ramdisk {
> > +                   compatible = "xen,linux-initrd", "xen,multiboot-module";
> > +                   reg = <0x0 0x3200000 0x0 $initrd>;
> > +           };
> > +   };
> > +};
> > +EOF
> > +dtc -I dts -O dtb virt.dts > virt.dtb
> > +
> > +rm -f smoke.serial
> > +set +e
> > +timeout -k 1 240 \
> > +./qemu-system-arm \
> > +   -machine virt-6.0 \
> > +   -machine virtualization=true \
> > +   -smp 4 \
> > +   -m 1024 \
> > +   -serial stdio \
> > +   -monitor none \
> > +   -display none \
> > +   -dtb virt.dtb \
> > +   -no-reboot \
> > +   -kernel ./xen \
> > +   -device loader,file=./vmlinuz,addr=0x1000000 \
> > +   -device loader,file=./initrd.gz,addr=0x3200000 |& tee smoke.serial
> > +
> > +set -e
> > +(grep -q "^/ #" smoke.serial) || exit 1
> > +exit 0
> 
> Cheers,
> Michal
>