Re: [PATCH 04/23] xen/arm: dom0less use domid 0 for hwdom

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Sat, 8 Mar 2025, Julien Grall wrote:
> On 08/03/2025 00:53, Stefano Stabellini wrote:
> > On Fri, 7 Mar 2025, Julien Grall wrote:
> > > > init-dom0less only initializes non- introduced domains, so hwdom doesn't
> > > > get
> > > > its "domid" xenstore node populated.  That leads to other errors.
> > > > > So I think with Denis's patch, this isn't strictly needed.  It does
> > > > > help
> > > > existing toolstack code work today.
> > > 
> > > I don't think the toolstack is ready for a split between control/hardware
> > > domain. That said, shouldn't the toolstack run in the control domain? Same
> > > for
> > > xenstored (unless you have a xenstored domain)?
> > 
> > Yes, the toolstack (if present) would be in the control domain.
> > xenstored doesn't have to be in the control domain and in fact it might
> > not be advisable to place it there today.
> > 
> > The main difference between the toolstack and xenstored is that the
> > toolstack only talks to Xen, while xenstored talks to all other VMs,
> > which is dangerous in many configurations.
> 
> It is not really clear which toolstack you are referring to.

You make a good point, I'll be clearer and distinguish between the
implementation and the interfaces. This exercise helped me clarify my
own thoughts as well.


> Someone using vanilla Xen upstream will end up to use "xl"  which has to talk
> to xenstored and also indirectly to each domain (e.g. shutdown/suspend node in
> xenstored). So for this setup, "xenstored" is not optional and I would argue
> should be part of the control domain (or in a xenstore stubdomain which IIRC
> is not supported on Arm today).

The control domain is the guest with the highest level of privilege and
the only virtual machine that can use hypercalls to affect other VMs. A
simple example is that the control domain could issue a hypercall to
reboot other VMs.

The hardware domain does not have privilege (d->is_priv=false).

In the configuration for safety, we use dom0less for domain creation, so
there is no toolstack is the traditional sense of the word. But we could
have a Zephyr guest issuing a SHUTDOWN_poweroff hypercall for instance.
Certainly, we cannot run Linux and xl/libxl in the control domain.

Due to its high level of privilege and the fact that it can affect
others, if present, the control domain must be safety certified. The
current toolstack implementation is based on Linux. While the larger
Open Source community is working on Linux for safety, it is still a
challenge. xenstored itself would need to be safety certified as well to
run there, think of MISRA C, and also requirements, tests, etc. In
addition, xenstored communicates directly with other VMs over shared
memory, which means it would need to be developed with extreme care to
prevent interference through shared memory. Even if a safety certified
version of Linux were available in the future, the current toolstack
implementation might not be suitable for use in that environment. As of
today, safety deployments would not be able to run xl/libxl and/or
xenstored in the control domain. However, they could run a safety
certified RTOS, maybe Zephyr, issuing individual hypercalls, such as
SHUTDOWN_reboot.

If we had a safety certified free-from-interference implementation of
xenstored as a Zephyr application, with Zephyr safety-certified, then I
would run it in the control domain.

On the other hand, the hardware domain might not be safety certified. So
we could run xl/libxl and xenstored in the hardware domain. Of course
most operations will be refused, e.g. SHUTDOWN_poweroff returns error
because d->is_priv=false, but at least the user could run today's
implementation of xenstored successfully. (domain creation is done using
dom0less anyway so xl is optional.)

Conceptually, it could make sense to think of xenstored being part of
the control domain, not today's xenstored but a safe re-implementation
of xenstored based on Zephyr.

If you look at what we have today available and what people could deploy
with the current codebase this year or next year, it makes sense to
place xenstored in the hardware domain, or not at all.