Re: [RFC PATCH] xen/design: Add design for EFI dom0less system start

[Julien Grall <julien@xxxxxxx>]

On 07/09/2021 12:51, Luca Fancellu wrote:
> > On 7 Sep 2021, at 10:35, Julien Grall <julien@xxxxxxx> wrote:
> >
> > Hi Luca,
> >
> > On 07/09/2021 07:52, Luca Fancellu wrote:
> > > Add a design describing a proposal to improve the EFI
> > > configuration file, adding keywords to describe domU
> > > guests and allowing to start a dom0less system.
> > > Signed-off-by: Luca Fancellu <luca.fancellu@xxxxxxx>
> > > ---
> > >   docs/designs/efi-arm-dom0less.md | 105 +++++++++++++++++++++++++++++++
> > >   1 file changed, 105 insertions(+)
> > >   create mode 100644 docs/designs/efi-arm-dom0less.md
> > > diff --git a/docs/designs/efi-arm-dom0less.md b/docs/designs/efi-arm-dom0less.md
> > > new file mode 100644
> > > index 0000000000..8d8fa2243f
> > > --- /dev/null
> > > +++ b/docs/designs/efi-arm-dom0less.md
> > > @@ -0,0 +1,105 @@
> > > +# Xen EFI configuration file
> > > +
> > > +The current configuration file used by Xen when it is started as an EFI
> > > +application is considering only the dom0 guest and doesn't have any
> > > +property to describe and load in memory domU guests.
> >  From my understanding, the problem is less about properties (we already have 
> > them in the Device-Tree) but more about where are the binaries located in 
> > memory as we don't know in advance.
> Hi Julien,
Hi Luca,

> I think I used the wrong word there, I meant “keyword” instead of “property” 
> because I was referring about the
> lack of keywords to describe a domu guest in the Xen EFI configuration file.
>
> I agree with you that on systems with static allocation, the kernel and ramdisk 
> binaries must be at certain locations
> that are out of control when we use the EFI boot services, the thing we can do 
> is provide a keyword to specify the
> addresses and then use the CopyMem() function to relocate the kernel/ramdisk in 
> the address we want.
I wasn't specifically referring to static allocation here, sorry if this 
wasn't clear. I was pointing out that most of the information you create 
in the xen.cfg is going to be similar to what we already provide in the 
Device-Tree.
My main concern is everytime we add a new feature in Dom0less, a 
developer would need to write code for the DT and UEFI. This will 
increase the code size and maintenance.
The same can be said for the admin as if they want to switch from plain 
U-boot to UEFI, they would also need to fully rewrite the bindings.
AFAICT, most of the information provided in the Device-Tree are usable 
even when using UEFI. So I would prefer if we try to re-use what's 
existing. This is what my proposal below was about.
> > So I would like to propose something that build on top of the Device-Tree work 
> > we did. Note this is early thoughts.
> >
> > The problematic nodes in the DT are:
> >
> >         module@0x4a000000 {
> >             compatible = "multiboot,kernel", "multiboot,module";
> >             reg = <0x0 0x4a000000 0xffffff>;
> >             bootargs = "console=ttyAMA0 init=/bin/sh";
> >         };
> >
> >         module@0x4b000000 {
> >             compatible = "multiboot,ramdisk", "multiboot,module";
> >             reg = <0x0 0x4b000000 0xffffff>;
> >         };
> >
> > In particular the property "reg" cannot be known in advance because the UEFI 
> > stub will be responsible to load the binaries in memory.
> Yes that’s true, the UEFI stub is using from the UEFI boot service the 
> AllocatePages function that is giving back an address out of our control,
> then using another function the binary is read from the disk and copied at that 
> address, finally the UEFI stub is writing the node in the device tree that
> will be used by Xen later.
I am not sure to follow. Are you saying the UEFI stub will create the 
dom0less node in the DT based on the xen.cfg?
> > What we could do is providing a list of binaries to load and associate a key 
> > for each of them. Something like:
> >
> > binary=<binary> <key>
> > binary=<binary2> <key2>
> > ....
> >
> > We can then replace the property "reg" with a new property "uefi,key" that will 
> > contain the name of the binary.
> >
> > What do you think?
> Here I’m lost, because I don’t understand what we are going to do with the name 
> of the binary.
<binaryX> would be used by the UEFI stub to load the binary in memory. 
Each binary will have a <keyX> which helps to refer them in the 
Device-Tree. To give a concrete example, let say we have two dom0less 
domains:
  - DomA: 2 vCPUs, 128MB
  - DomB: 3 vCPUs, 512MB

DomA and DomB will be using the same kernel but a different ramdisk. 
xen.cfg, would look like:
[global]
default=section1

[section1]
options=console=vga,com1 com1=57600 loglvl=all noreboot
kernel=vmlinuz-3.0.31-0.4-xen [domain 0 command line options]
ramdisk=initrd-3.0.31-0.4-xen
xsm=<filename>
dtb=devtree.dtb
binary=vmlinuz-guest domu-kernel
binary=ramdisk-domA.img domA-ramdisk
binary=ramdisk-domB.img domB-ramdisk

The chosen node in the DT would look like:

chosen {
    domU1 {
        compatible = "xen,domain";
        #address-cells = <0x2>;
        #size-cells = <0x1>;
        memory = <0 0x8000000>;
        cpus = <2>;

        module@1 {
            compatible = "multiboot,kernel", "multiboot,module";
            uefi,binary = "domu-kernel";
            bootargs = "console=ttyAMA0 init=/bin/sh";
        };

        module@2 {
            compatible = "multiboot,ramdisk", "multiboot,module";
            uefi,binary = "domA-ramdisk";
        };
    };

    domU2 {
        compatible = "xen,domain";
        #address-cells = <0x3>;
        #size-cells = <0x1>;
        memory = <0 0x20000000>;
        cpus = <3>;

        module@1 {
            compatible = "multiboot,kernel", "multiboot,module";
            uefi,binary = "domu-kernel";
            bootargs = "console=ttyAMA0 init=/bin/sh";
        };

        module@2 {
            compatible = "multiboot,ramdisk", "multiboot,module";
            uefi,binary = "domA-ramdisk";
        };
    };
};

With this approach, the change is quite minimal to move between an 
classic U-boot boot and EFI boot.
Cheers,

--
Julien Grall