Re: xc_dom_guest_type: image not capable of booting inside a HV M container: Invalid kernel

[Chuck Zmudzinski <brchuckz@xxxxxxxxxxxx>]

On 12/11/2023 9:02 AM, Mario Marietto wrote:
> Hello.
> 
> Finally I tried to recompile the FreeBSD kernel using the @Elliott Mitchell 
> <mailto:ehem+freebsd@xxxxxxx> code because I want to boot FreeBSD as domU 
> with Xen installed on my Arm 32 bit Chromebook. Unfortunately it didn't work 
> at all. Maybe I've missed something / I haven't understood well what to do. 
> Please give me some suggestions.
> 
> Basically this is what I did :
> 
> $ created a vm called FreeBSD-13.2-RELEASE-armv7.img with qemu / kvm / 
> libvirt / virt-manager
> 
> $ within the vm : mkdir /build-xen
> 
> $ cd /usr
> 
> $ git clone https://gitlab.com/ehem/freebsd-src.git 
> <https://gitlab.com/ehem/freebsd-src.git?fbclid=IwAR09ukOdBXCR3gJznvHWpnWM8VEZqnX1l4ZMza_7lhoAjfPYAGbvcqwjh94>
> 
> $ cd freebsd-src
> 
> $ make KERNCONF=GENERIC TARGET=arm TARGET_ARCH=armv7 buildkernel
> 
> $ make KERNCONF=GENERIC TARGET=arm TARGET_ARCH=armv7 DESTDIR=/build-xen 
> installkernel
> 
> $ echo "/dev/xbd0 / ufs rw 1 1" > /mnt/etc/fstab
> 
> $ nano /etc/ttys (add the line 'xc0 "/usr/libexec/getty Pc" xterm on secure")
> 
> $ renamed the directories dtb to dtb_ and kernel to kernel_ that are inside 
> the /boot dir of the vm
> 
> $ copied the directory dtb and kernel from the directory /build-xen to the 
> directory /boot inside the vm
> 
> $ shut down the vm
> 
> $ copied the directory /build-xen outside of the vm using this method (in 
> this case I used Linux installed on the Host OS,because the kernel that I'm 
> using on the Chromebook has the kernel parameter related to the ufs2 fs set 
> to off) :
> 
> on my X64 workstation :
> 
> # modprobe ufs
> 
> # sudo losetup -fP FreeBSD-13.2-RELEASE-armv7.img
> 
> # ls /dev/loop0*
> 
> /dev/loop0 /dev/loop0p1 /dev/loop0p2 /dev/loop0p5
> 
> # mount -t ufs -o ufstype=ufs2 /dev/loop0p5 ./FreeBSD-xen
> 
> then :
> 
> # nano freebsd.cfg
> 
> kernel="/mnt/zroot2/zroot2/OS/Chromebook/domU/freebsd-xen/boot-xen/kernel/kernel"
> memory=64
> name="freebsd"
> vcpus=1
> autoballon="off"
> disk=[ 'phy:/dev/loop0,xvda,w' ]
> # nano start-freebsd
> losetup -fP FreeBSD-13.2-RELEASE-armv7.img
> xl create freebsd.cfg
> xl console freebsd
> 
> # ./start-freebsd
> 
> Parsing config from freebsd.cfg
> xc: error: panic: xg_dom_elfloader.c:63: xc_dom_guest_type: image not capable 
> of booting inside a HV
> M container: Invalid kernel

It is detecting the kernel as an elf binary. IIUC, Xen on arm guests should 
have zImage kernels, not elf.

> libxl: error: libxl_dom.c:571:libxl__build_dom: xc_dom_parse_image failed
> libxl: error: libxl_create.c:1640:domcreate_rebuild_done: Domain 1:cannot 
> (re-)build domain: -3
> libxl: error: libxl_domain.c:1183:libxl__destroy_domid: Domain 1:Non-existent 
> domain
> libxl: error: libxl_domain.c:1137:domain_destroy_callback: Domain 1:Unable to 
> destroy guest
> libxl: error: libxl_domain.c:1064:domain_destroy_cb: Domain 1:Destruction of 
> domain failed
> freebsd is an invalid domain identifier (rc=-6)
> 
> I have also tried with kernel.bin :
> 
> # nano freebsd.cfg
> 
> kernel="/mnt/zroot2/zroot2/OS/Chromebook/domU/freebsd-xen/boot-xen/kernel/kernel.bin"
> memory=64
> name="freebsd"
> vcpus=1
> autoballon="off"
> disk=[ 'phy:/dev/loop0,xvda,w' ]
> 
> # ./start-freebsd
> 
> Parsing config from freebsd.cfg
> xc: error: panic: xg_dom_core.c:689: xc_dom_find_loader: no loader found: 
> Invalid kernel
> libxl: error: libxl_dom.c:571:libxl__build_dom: xc_dom_parse_image failed
> libxl: error: libxl_create.c:1640:domcreate_rebuild_done: Domain 2:cannot 
> (re-)build domain: -3
> libxl: error: libxl_domain.c:1183:libxl__destroy_domid: Domain 2:Non-existent 
> domain
> libxl: error: libxl_domain.c:1137:domain_destroy_callback: Domain 2:Unable to 
> destroy guest
> libxl: error: libxl_domain.c:1064:domain_destroy_cb: Domain 2:Destruction of 
> domain failed
> freebsd is an invalid domain identifier (rc=-6)
> 
> -- 
> Mario.

I would be interested to see the output of :

$ file /mnt/zroot2/zroot2/OS/Chromebook/domU/freebsd-xen/boot-xen/kernel/kernel

and

$ file 
/mnt/zroot2/zroot2/OS/Chromebook/domU/freebsd-xen/boot-xen/kernel/kernel.bin

I have been trying out Julien's old patch set from 2014, and in there was this 
patch :

> arm: Add zImage support
> 
> Currently Xen on ARM is only supported zImage for guest kernel. Adding support
> for ARM ELF in the toolstack looks a bit complicate for ARM (though there is
> an x86 support).

Link to Julien's 2014 patch to provide zImage support for FreeBSD :

https://xenbits.xen.org/gitweb/?p=people/julieng/freebsd.git;a=commit;h=12a7cb346b88c6d3f52a20b98f361dc62797fbcd

When using Julien's patches, from 'file' I find that the kernel file is in
the elf format, and the kernel.bin file is in the zImage format, so I have
been trying to boot the kernel.bin file.