Re: request for feedback on a Xen/Linux compatibility issue

[Julien Grall <julien@xxxxxxx>]

Hi,

On 06/01/2022 14:03, Jan Beulich wrote:
> On 06.01.2022 08:13, Juergen Gross wrote:
> > On 06.01.22 01:40, Stefano Stabellini wrote:
> > > Hi all,
> > >
> > > Today Xen dom0less guests are not "Xen aware": the hypervisor node
> > > (compatible = "xen,xen") is missing from dom0less domUs device trees and
> > > as a consequence Linux initializes as if Xen is not present. The reason
> > > is that interfaces like grant table and xenstore (xenbus in Linux) don't
> > > work correctly in a dom0less environment at the moment.
> > >
> > > The good news is that I have patches for Xen to implement PV drivers
> > > support for dom0less guests. They also add the hypervisor node to device
> > > tree for dom0less guests so that Linux can discover the presence of Xen
> > > and related interfaces.
> > >
> > > When the Linux kernel is booting as dom0less kernel, it needs to delay
> > > the xenbus initialization until the interface becomes ready. Attempts to
> > > initialize xenbus straight away lead to failure, which is fine because
> > > xenbus has never worked in Linux when running as dom0less guest up until
> > > now. It is reasonable that a user needs a newer Linux to take advantage
> > > of dom0less with PV drivers. So:
> > >
> > > - old Xen + old/new Linux -> Xen not detected in Linux
> > > - new Xen + old Linux     -> xenbus fails to initialize in Linux
> > > - new Xen + new Linux     -> dom0less PV drivers working in Linux
> > >
> > >
> > > The problem is that Linux until recently couldn't deal with any errors
> > > in xenbus initialization. Instead of returning error and continuing
> > > without xenbus, Linux would crash at boot.
> > >
> > > I upstreamed two patches for Linux xenbus_probe to be able to deal with
> > > initialization errors. With those two fixes, Linux can boot as a
> > > dom0less kernel with the hypervisor node in device tree. The two fixes
> > > got applied to master and were already backported to all the supported
> > > Linux stable trees, so as of today:
> > >
> > > - dom0less with hypervisor node + Linux 5.16+           -> works
> > > - dom0less with hypervisor node + stable Linux 5.10     -> works
> > > - dom0less with hypervisor node + unpatched Linux 5.10  -> crashes
> > >
> > >
> > > Is this good enough? Or for Xen/Linux compatibility we want to also be
> > > able to boot vanilla unpatched Linux 5.10 as dom0less kernel? If so,
> > > the simplest solution is to change compatible string for the hypervisor
> > > node, so that old Linux wouldn't recognize Xen presence and wouldn't try
> > > to initialize xenbus (so it wouldn't crash on failure). New Linux can of
> > > course learn to recognize both the old and the new compatible strings.
> > > (For instance it could be compatible = "xen,xen-v2".) I have prototyped
> > > and tested this solution successfully but I am not convinced it is the
> > > right way to go.
> > >
> > > Do you have any suggestion or feedback?
> > >
> > > The Linux crash on xenbus initialization failure is a Linux bug, not a
> > > Xen issue. For this reason, I am tempted to say that we shouldn't change
> > > compatible string to work-around a Linux bug, especially given that the
> > > Linux stable trees are already all fixed.
> > What about adding an option to your Xen patches to omit the hypervisor
> > node in the device tree? This would enable the user to have a mode
> > compatible to today's behavior.
> While this sounds nice at the first glance, this would need to be a per-
> domain setting. Which wouldn't be straightforward to express via command
> line option (don't know how feasible it would be to express such via other
> means).
For dom0less, domains are described in the Device-Tree. We have one node 
per domain, so we could add a property to indicate whether the domain 
should be started in compat mode (or not).
That said, I am not sure every users will want Linux to use 
grant-table/xenstore (possibly, some users may want one but not the other).
So how about a more generic property "xen,enhanced" with an opional 
value indicating whether this is disabled, enabled or the list of 
interface (e.g. xenbus, grant-table) exposed?
Cheers,

--
Julien Grall