Re: Revisit: HVM on storage driver domain

[Kuba <kuba.0000@xxxxx>]

> Revisiting a failed attempt 5 years ago but still don't have the luck :-(
>
> >> BTW, an irrelevant question:
> >> What's the current status of HVM domU on top of storage driver domain?
> >> About 7 years ago, one user on the list was able to get this setup up
> >> and running with your help (patch).[1]
> >> When I attempted to reproduce a similar setup two years later, I
> >> discovered that the patch was not submitted.
> >> And even with that patch the setup cannot be reproduced successfully.
> >> We spent some time debugging on the problem together[2], but didn't
> >> bottom out the root cause at that time.
> >> In case it's still broken and you still have the interest and time, I
> >> can launch a separate thread on this topic and provide required
> >> testing environment.
> >
> >  Yes, better as a new thread please.
> >
> > FWIW, I haven't looked at this since a long time, but I recall some
> > fixes in order to be able to use driver domains with HVM guests, which
> > require attaching the disk to dom0 in order for the device model
> > (QEMU) to access it.
> >
> > I would give it a try without using stubdomains and see what you get.
> > You will need to run `xl devd` inside of the driver domain, so you
> > will need to install xen-tools on the domU. There's an init script to
> > launch `xl devd` at boot, it's called 'xendriverdomain'.
> >
> For this testing purpose, I did the following:
> 1. downloaded an official FreeBSD 12.2 VM image as a domU
> 2. Install the xen-tools package and enable the devd daemon
> 3. reboot the domU with driver_domain=1, mount the guest image from NAS
> 4. instance a new domU with the following config:
>      device_model_stubdomain_override=0
>      disk = 
> ['backend=freebsd12,/mnt/vmfs/Windows/ruibox/ruibox.img,raw,xvda,w']
> 5. When boot the new domU, see the following warning and the firmware 
> cannot boot the disk:
>        libxl: warning: 
> libxl_dm.c:1888:libxl__build_device_model_args_new: Domain 17:No way 
> to get local access disk to image: xvda
>        Disk will be available via PV drivers but not as anemulated disk.
> 6. When boot with stubdomain_override=1, it fails differently:
>     libxl: error: libxl_dm.c:2332:libxl__spawn_stub_dm: Domain 
> 79:could not access stubdomain kernel 
> /usr/lib/xen-4.14/boot/ioemu-stubdom.gz: No such file or directory
>     libxl: error: libxl_dm.c:2724:stubdom_pvqemu_cb: Domain 79:error 
> connecting nics devices: No such file or directory
> Looks like the XEN 4.14 package from Debian does not have the stubdom 
> feature available.
> I wonder if it's just a single file that I can grab from somewhere or 
> if I need to build xen from the source to get this stubdom work?
> Anything obviously wrong in my setup, or suggestion on diagnosis, Roger?
>
> Thanks,
> G.R.
>
> > Thanks, Roger.
> >
> >> [1] 
> https://lists.xenproject.org/archives/html/xen-users/2014-08/msg00003.html
> >> [2] 
> https://xen-users.narkive.com/9ihP0QG4/hvm-domu-on-storage-driver-domain
Hi,

I remember using Roger's patch quite successfully not that long ago with 
some 4.10+ Xen (it even might have been 4.14, but I am not sure).
It requires compiling Xen from source, but as a result you get a deb 
package. After that all you need is to use the config like in the [1] 
link above and it "just works".
I don't guarantee anything, but I think it is worth to try.

Regards,
Kuba