request for feedback on a Xen/Linux compatibility issue

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

Hi all,

Today Xen dom0less guests are not "Xen aware": the hypervisor node
(compatible = "xen,xen") is missing from dom0less domUs device trees and
as a consequence Linux initializes as if Xen is not present. The reason
is that interfaces like grant table and xenstore (xenbus in Linux) don't
work correctly in a dom0less environment at the moment.

The good news is that I have patches for Xen to implement PV drivers
support for dom0less guests. They also add the hypervisor node to device
tree for dom0less guests so that Linux can discover the presence of Xen
and related interfaces.

When the Linux kernel is booting as dom0less kernel, it needs to delay
the xenbus initialization until the interface becomes ready. Attempts to
initialize xenbus straight away lead to failure, which is fine because
xenbus has never worked in Linux when running as dom0less guest up until
now. It is reasonable that a user needs a newer Linux to take advantage
of dom0less with PV drivers. So:

- old Xen + old/new Linux -> Xen not detected in Linux
- new Xen + old Linux     -> xenbus fails to initialize in Linux
- new Xen + new Linux     -> dom0less PV drivers working in Linux


The problem is that Linux until recently couldn't deal with any errors
in xenbus initialization. Instead of returning error and continuing
without xenbus, Linux would crash at boot.

I upstreamed two patches for Linux xenbus_probe to be able to deal with
initialization errors. With those two fixes, Linux can boot as a
dom0less kernel with the hypervisor node in device tree. The two fixes
got applied to master and were already backported to all the supported
Linux stable trees, so as of today:

- dom0less with hypervisor node + Linux 5.16+           -> works
- dom0less with hypervisor node + stable Linux 5.10     -> works
- dom0less with hypervisor node + unpatched Linux 5.10  -> crashes


Is this good enough? Or for Xen/Linux compatibility we want to also be
able to boot vanilla unpatched Linux 5.10 as dom0less kernel? If so,
the simplest solution is to change compatible string for the hypervisor
node, so that old Linux wouldn't recognize Xen presence and wouldn't try
to initialize xenbus (so it wouldn't crash on failure). New Linux can of
course learn to recognize both the old and the new compatible strings.
(For instance it could be compatible = "xen,xen-v2".) I have prototyped
and tested this solution successfully but I am not convinced it is the
right way to go.

Do you have any suggestion or feedback?

The Linux crash on xenbus initialization failure is a Linux bug, not a
Xen issue. For this reason, I am tempted to say that we shouldn't change
compatible string to work-around a Linux bug, especially given that the
Linux stable trees are already all fixed.

Cheers,

Stefano