Xen project Mailing List

On Tue, Nov 28, 2023 at 11:41 AM Mario Marietto <marietto2008@xxxxxxxxx> wrote:

Hello.

to try to compile the correct freebsd kernel that's recognized by xen using the Elliott Michell code, are the commands below the correct ones ?
$ truncate -s 100G xenvm.img
$ mdconfig -f xenvm.img -u 0 
$ newfs /dev/md0 
$ mount /dev/md0 /mnt
$ git clone https://gitlab.com/ehem/freebsd-src.git
$ cd freebsd-src

$ make  -DNO_MODULES   KERNCONF=GENERIC TARGET=arm TARGET_ARCH=armv7 buildkernel

$ echo "/dev/xbd0       /       ufs     rw      1 
1" > /mnt/etc/fstab  

$ nano /build/etc/ttys (add the line 'xc0 "/usr/libexec/getty Pc" xterm on secure")
$ umount /build
$ mdconfig -d -u 0
Do you see errors ? some missing ? very thanks.
On Tue, Nov 28, 2023 at 3:28 AM Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:
On Mon, 27 Nov 2023, Chuck Zmudzinski wrote:
> On 11/27/2023 10:22 AM, Chuck Zmudzinski wrote:
> > On 11/27/2023 7:45 AM, Mario Marietto wrote:
> >> @Chuck Zmudzinski <mailto:brchuckz@xxxxxxxxxxxx> : Stay tuned. They want to help us. The xen developers are great. Very good support for us. I'm sure that you can give a good contribution to understand what's our problem and how to implement a fix with the help of all those good guys.
> >>
> >> On Mon, Nov 27, 2023 at 11:56 AM Roger Pau Monné <roger.pau@xxxxxxxxxx <mailto:roger.pau@xxxxxxxxxx>> wrote:
> >>
> >> On Mon, Nov 27, 2023 at 10:28:13AM +0000, Henry Wang wrote:
> >> > +(xen-devel and Arm maintainers, including Julien)
> >> >
> >> > > On Nov 27, 2023, at 18:03, Mario Marietto <marietto2008@xxxxxxxxx <mailto:marietto2008@xxxxxxxxx>>
> >> > > wrote:
> >> > >
> >> > > Hello. We have just virtualized Debian 12 on our arm (32 bit)
> >> > > Chromebook model xe303c12 . As host / dom0 we have chosen Devuan
> >> > > 5,and for guest / domU,Debian 12. It works great. But our goal is
> >> > > different. We want to virtualize FreeBSD as domU. Can we have a
> >> > > working Xen PV network driver for a FreeBSD arm guest ?. I found
> >> > > that Julien Grall has ported the Xen drivers to FreeBSD on arm. I
> >> > > would like to know if Julien's work was accepted upstream by
> >> > > FreeBSD, in which case FreeBSD as a Xen guest on arm should work
> >> > > if we enable the Xen PV drivers in the FreeBSD on arm kernel. If
> >> > > Julien's work was not accepted upstream by FreeBSD, we will have
> >> > > to find his patches and apply them ourselves to the FreeBSD on arm
> >> > > kernel.
> >>
> >> I've added Elliot on Cc as he is working on upstreaming the patches to
> >> FreeBSD. He will be able to provide a better update than myself.
> >>
> >> Regards, Roger.
> >
> > I have been collaborating with Mario, and I can explain what we have done so far :
> >
> > We are using Julien's patch set against an old development version of FreeBSD 11
> > from 2014-12-03 :
> >
> > https://xenbits.xen.org/gitweb/?p=people/julieng/freebsd.git;a=shortlog;h=refs/heads/xen-arm-v2.2
> >
> > We successfully built the XENVIRT kernel and FreeBSD world, and created the
> > FreeBSD rootfs according to Julien's instructions here :
> >
> > https://lists.freebsd.org/pipermail/freebsd-xen/2014-November/002202.html
> >
> > There were some adjustments to the instructions :
> >
> > To build the kernel, we used :
> >
> > $ sudo make TARGET_ARCH=armv6 KERNCONF=XENVIRT buildkernel
> >
> > instead of
> >
> > $ sudo make TARGET_ARCH=armv6 KERNCONF=XENHVM buildkernel
> >
> > The FreeBSD 'kernel' file is in ELF format and did not work, and we spent
> > some time trying to convert it to the zImage format without realizing the
> > build of the FreeBSD kernel creates the 'kernel.bin' file in the zImage format.
> > So when booting with the 'kernel.bin' file instead, it actually boots :
> >
> > user@devuan-bunsen ~ % sudo xl create freebsd.cfg
> > Parsing config from freebsd.cfg
> > user@devuan-bunsen ~ % sudo xl li
> > Name ID Mem VCPUs State Time(s)
> > Domain-0 0 768 2 r----- 1439.4
> > freebsd 1 1152 1 r----- 3.0
> > user@devuan-bunsen ~ %
> >
> > However, the guest is still not working correctly :
> >
> > 1. Attaching the console with the -c option at creation or with
> > 'xl console freebsd' results in no output to the console.
> >
> > 2. The timestamp on the virtual disk image file shows that the filesystem
> > was at best mounted read-only, if it was mounted at all by the guest
> > FreeBSD kernel.
> >
> > 3. The 'xl shutdown freebsd' command does not work, it just hangs. To stop
> > the guest, you need to do 'xl destroy freebsd'.
> >
> > However, I think we can get the console to work and the rootfs to mount because I
> > just realized I forgot to do the steps from Julien's instructions of editing the
> > /etc/fstab and /etc/ttys files in the FreeBSD rootfs :
> >
> > $ echo "/dev/xbd0 / ufs rw 1 1" > /mnt/etc/fstab
> > $ vi /mnt/etc/ttys (add the line 'xc0 "/usr/libexec/getty Pc" xterm on secure")
> >
> > I will add those and see if the console and disk are working.
>
> Unfortunately, adding xc0 to /etc/ttys and /dev/xbd0 as the root device in
> /etc/fstab did not make the console or disk work. Still no output on the
> xen console from the guest kernel, and the timestamp on the rootfs image
> file did not change so it did not mount read-write.
>
> We could use some advice for troubleshooting this. Now, we are blind because
> we are not getting any xen console output But I am pleased we were able to
> demonstrate that Julien's old patch set for FreeBSD 11 allows us to boot
> FreeBSD 11 on a modern version of Xen on arm - we are using the Debian
> stable Xen 4.17 packages.

You can use the DEBUG hypercalls to check how far we got into the
booting process:
https://wiki.xenproject.org/wiki/Xen_ARM_DEBUG_hypercalls

For instance add the following to FreeBSD code:

asm volatile("hvc 0xfffd");

--
Mario.

Re: We are not able to virtualize FreeBSD using xen 4.17 on Arm 32 bit