Re: We are not able to virtualize FreeBSD using xen 4.17 on Arm 32 bit

[Mario Marietto <marietto2008@xxxxxxxxx>]

Hello to everyone.

I tried to use xen as a hypervisor instead of kvm + libvirt + virt-manager to boot FreeBSD on my ARM Chromebook where I have installed Devuan 5,since Stefano said : 

"That might work. libvirt + virt-manager with the xen accelerator might work on the ARM Chromebook. That's because as far as I know Xen integration in libvirt is done via linking to libxl directly and libxl is supported and working on ARM" 

Unfortunately something is not working properly. What I did has been to reboot the machine in xen,enable libvirtd & and virtlogd & and virt-manager &,but this is what happened :

Traceback (most recent call last):
  File "/usr/lib/xen-4.17/bin/pygrub", line 884, in <module>
    raise RuntimeError("Unable to find partition containing kernel")
RuntimeError: Unable to find partition containing kernel

I think it does not recognize the FreeBSD file system structure and its kernel. Libvirt seems to have been programmed to boot Linux,not FreeBSD. 

In Fact,I did the counterproof and it seems to be like this :

According with this post : https://blog.roberthallam.org/2020/05/solving-unable-to-find-partition-containing-kernel/ I've created a file called menu.lst inside the boot directory of the image file called "debian.img",adding the following content inside : default 0 timeout 10 title Debian root (hd0,1) kernel /boot/vmlinux-6.1.59-stb-xen-cbe+ root=/dev/xvda initrd /boot/initrd.img-6.1.59-stb-xen-cbe+ and I tried again to boot the image using virt-manager. It gave this error again : root@devuan-bunsen:/mnt/zroot2/zroot2/OS/Chromebook/FreeBSD-guestOS/linux-xen/debian2/boot# 2023-11-29 15:21:09.266+0000: 2467: error : libxlDomainStartPerform:1256 : internal error: libxenlight failed to create new domain 'debian12' but giving a look inside the log file and I found this interesting situation ; Using <class 'grub.GrubConf.GrubConfigFile'> to parse /boot/grub/menu.lst (B )0 [1;24r [m [?7h [?1h = [H [J [?1h = [1B    pyGRUB  version 0.6 [1B [0m lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk [1B [0m x [0;7m Debian 12                                                             [m [0m x [1B [0m x [72C [0m x [1B [0m x [72C [0m x [1B [0m x [72C [0m x [1B [0m x [72C [0m x [1B [0m x [72C [0m x [1B [0m x [72C [0m x [1B [0m x [72C [0m x [1B [0m mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj [1B [70D [0m Use the ↑ and ↓ keys to select which entry is highlighted. [1B [58DPress enter to boot the selected OS, 'e' to edit the [1B [52Dcommands before booting, 'a' to modify the kernel arguments [1B [59Dbefore booting, or 'c' for a command line. [12A [26C [17B [68DWill boot selected entry in 10 seconds [?1h = [J [17A [73C [17B [68DWill boot selected entry in  9 seconds [?1h = [J [17A [73C [17B [68DWill boot selected entry in  8 seconds [?1h = [J [17A [73C [17B [68DWill boot selected entry in  7 seconds [?1h = [J [17A [73C [17B [68DWill boot selected entry in  6 seconds [?1h = [J [17A [73C [17B [68DWill boot selected entry in  5 seconds [?1h = [J [17A [73C [17B [68DWill boot selected entry in  4 seconds [?1h = [J [17A [73C [17B [68DWill boot selected entry in  3 seconds [?1h = [J [17A [73C [17B [68DWill boot selected entry in  2 seconds [?1h = [J [17A [73C [17B [68DWill boot selected entry in  1 seconds [?1l > [24;1H [?1l > so,it seems that it tried to boot,but for an unknown reason,it still gives the error.   Anyway : My xen setup is not broken anymore ; Using Libvirt Linux seems to be able to boot,FreeBSD does not. Using the "raw" method of booting FreeBSD as domU could be another story that I will try soon.

But before trying to compile the correct freebsd kernel that's recognized by xen using the Elliott Michell code,I need to understand well what's the procedure that will work. So below you can read what I will try to do :

$ truncate -s 100G xenvm.img
$ mdconfig -f xenvm.img -u 0 
$ newfs /dev/md0 
$ mount /dev/md0 /mnt
$ git clone https://gitlab.com/ehem/freebsd-src.git
$ cd freebsd-src

$ make  -DNO_MODULES   KERNCONF=GENERIC TARGET=arm TARGET_ARCH=armv7 DESTDIR=/build buildkernel

$ echo "/dev/xbd0       /       ufs     rw      1 
1" > /mnt/etc/fstab  

$ nano /build/etc/ttys (add the line 'xc0 "/usr/libexec/getty Pc" xterm on secure")
$ umount /build
$ mdconfig -d -u 0

Do you see errors ? some missing ?


---> I've only ever tried arm64, but since arm32 didn't appear to need much to be operational I tried to
make it possible.  In theory it /should/ work on arm32, but I've never tried it.  What was missing was 
I had never added it to the configuration and one link was needed. Updated "submit" branch has a tiny 
adjustment.

I didn't understand how to apply the patch that's inside the "submit" branch. 

On Wed, Nov 29, 2023 at 12:05 AM Elliott Mitchell <ehem+freebsd@xxxxxxx> wrote:

On Tue, Nov 28, 2023 at 04:10:50PM +0100, Roger Pau Monné wrote:
> On Tue, Nov 28, 2023 at 03:09:14PM +0100, Mario Marietto wrote:
> > For booting a FreeBSD kernel as a guest OS on XEN,should we install xen
> > 4.18 from source ?

> I don't think so, I'm not aware of the FreeBSD port requiring a
> specific version of Xen.  I do think the work is limited to aarch64
> however, so there's no support in sight for arm32 FreeBSD guests as
> far as I'm aware.

I've only ever tried arm64, but since arm32 didn't appear to need much
to have operational I tried to make it possible.  In theory it /should/
work on arm32, but I've never tried it.  What was missing was I had never
added it to the configuration and one link was needed.  Updated "submit"
branch has the tiny adjustment.

(the only difference is the hypercall wrappers, register naming and where
the op code goes, very simple compatibility)


On Tue, Nov 28, 2023 at 02:45:40PM +0100, Roger Pau Monné wrote:
> On Mon, Nov 27, 2023 at 03:04:30PM -0800, Elliott Mitchell wrote:
> > BTW Roger Pau Monné, now that Xen 4.18 is out, take a look at the
> > "royger" branch?
>
> I've pushed a bunch of those, there are still some, I've made comments
> on the branch.
>
> I think there isn't much left after the swept I've done.
>
> If you can rebase and reply to the comments I will take a look at
> what's remaining.

Done.  I'm unsure you'll like the xs_attach_children() approach.  Thing
is that really is appropriate given the situation.  #2 is the urgent one
as that is the handy approach to the hypercall declarations.


--
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@xxxxxxx  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445

--

Mario.