Re: [PATCH 1/3] Rework rootfs generation to make a cpio archive

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 11/04/2025 1:25 pm, Frediano Ziglio wrote:
> On Fri, Apr 11, 2025 at 11:52 AM Andrew Cooper
> <andrew.cooper3@xxxxxxxxxx> wrote:
>> Rename the script as we're going to use it for ARM64 shortly, and have it 
>> take
>> a tar or cpio parameter to determine the output format.
>>
>> Turn it into a proper bash script, and provide the cpio form under the new
>> artefact naming scheme.
>>
>> No functional change.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
>> ---
>> CC: Anthony PERARD <anthony.perard@xxxxxxxxxx>
>> CC: Stefano Stabellini <sstabellini@xxxxxxxxxx>
>> CC: Michal Orzel <michal.orzel@xxxxxxx>
>> CC: Doug Goldstein <cardoe@xxxxxxxxxx>
>> CC: Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>
>> ---
>>  .gitlab-ci.yml                                |  9 +++++++-
>>  ...6_64-rootfs-alpine.sh => alpine-rootfs.sh} | 21 +++++++++++++++++--
>>  2 files changed, 27 insertions(+), 3 deletions(-)
>>  rename scripts/{x86_64-rootfs-alpine.sh => alpine-rootfs.sh} (75%)
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> index 1d2c72b268a3..916c5ae9d508 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -42,6 +42,13 @@ linux-6.6.86-arm64:
>>  #
>>  # x86_64 artifacts
>>  #
>> +alpine-3.18-x86_64-rootfs:
>> +  extends: .x86_64-artifacts
>> +  script:
>> +    - ./scripts/alpine-rootfs.sh cpio
>> +  variables:
>> +    CONTAINER: alpine:3.18-x86_64-base
>> +
>>  linux-6.6.56-x86_64:
>>    extends: .x86_64-artifacts
>>    script: ./scripts/build-linux.sh
>> @@ -62,7 +69,7 @@ x86_64-kernel-linux-6.6.56:
>>  x86_64-rootfs-alpine-3.18:
>>    extends: .x86_64-artifacts
>>    script:
>> -    - . scripts/x86_64-rootfs-alpine.sh
>> +    - ./scripts/alpine-rootfs.sh tar
>>    variables:
>>      CONTAINER: alpine:3.18-x86_64-base
>>
>> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/alpine-rootfs.sh
>> similarity index 75%
>> rename from scripts/x86_64-rootfs-alpine.sh
>> rename to scripts/alpine-rootfs.sh
>> index b70b3a54ede5..c29c92d1c682 100755
>> --- a/scripts/x86_64-rootfs-alpine.sh
>> +++ b/scripts/alpine-rootfs.sh
>> @@ -1,4 +1,9 @@
>> +#!/bin/bash
>> +
>> +set -eu
>> +
>>  WORKDIR="${PWD}"
>> +COPYDIR="${WORKDIR}/binaries"
>>
>>  apk update
>>
>> @@ -56,5 +61,17 @@ passwd -d "root" root
>>
>>  # Create rootfs
>>  cd /
>> -tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
>> -    bin dev etc home init lib mnt opt root sbin usr var
>> +PATHS="bin dev etc home init lib mnt opt root sbin usr var"
>> +
>> +case $1 in
>> +    cpio)
>> +        find $PATHS | cpio -o -H newc | gzip > "${COPYDIR}/rootfs.cpio.gz"
>> +
> Paranoia mode: -print0 for find and -0 for cpio to support weird file names?

This is a tiny alpine rootfs.  There aren't any, but weird filenames
would be a problem elsewhere too.

>
>> +        # Print the contents for the build log
>> +        zcat "${COPYDIR}/rootfs.cpio.gz" | cpio -tv
> I saw other comments about -v and busybox compatibility.
> I suppose it depends how important are performances here, otherwise I
> could suggest the usage of mkfifo before, use -F option to the fifo
> and use an additional pipeline to compress the file.
>
> mypipe=$(mktemp -u)
> mkfifo $mypipe
> gzip -c > "${COPYDIR}/rootfs.cpio.gz" < $mypipe &
> find $PATHS -print0 | cpio -o -H newc -F $mypipe -0
> wait # for gzip
> rm $mypipe

These artefacts are built once, then reused indefinitely.  Generating
this artefact takes 18s, most of which is the gitlab runner starting up.

~Andrew