Re: [Xen-devel] [RFC] ARM VM System Sepcification

[Christoffer Dall <christoffer.dall@xxxxxxxxxx>]

On Wed, Feb 26, 2014 at 08:55:58PM +0100, Arnd Bergmann wrote:
> On Wednesday 26 February 2014 10:34:54 Christoffer Dall wrote:
> > ARM VM System Specification
> > ===========================
> > 
> > Goal
> > ----
> > The goal of this spec is to allow suitably-built OS images to run on
> > all ARM virtualization solutions, such as KVM or Xen.
> > 
> > Recommendations in this spec are valid for aarch32 and aarch64 alike, and
> > they aim to be hypervisor agnostic.
> > 
> > Note that simply adhering to the SBSA [2] is not a valid approach,
> > for example because the SBSA mandates EL2, which will not be available
> > for VMs.  Further, the SBSA mandates peripherals like the pl011, which
> > may be controversial for some ARM VM implementations to support.
> > This spec also covers the aarch32 execution mode, not covered in the
> > SBSA.
> 
> I would prefer if we can stay as close as possible to SBSA for individual
> hardware components, and only stray from it when there is a strong reason.
> pl011-subset doesn't sound like a significant problem to implement,
> especially as SBSA makes the DMA part of that optional. Can you
> elaborate on what hypervisor would have a problem with that?
> 

The Xen guys are hard-set on not supporting a pl011.  If we can convince
them or force it upon them, I'm ok with that.

I agree we should stay close to the SBSA, but I think there are
considerations for VMs beyond the SBSA that warrants this spec.

> > Hardware Description
> > --------------------
> > The Linux kernel's proper entry point always takes a pointer to an FDT,
> > regardless of the boot mechanism, firmware, and hardware description
> > method.  Even on real hardware which only supports ACPI and UEFI, the kernel
> > entry point will still receive a pointer to a simple FDT, generated by
> > the Linux kernel UEFI stub, containing a pointer to the UEFI system
> > table.  The kernel can then discover ACPI from the system tables.  The
> > presence of ACPI vs. FDT is therefore always itself discoverable,
> > through the FDT.
> > 
> > Therefore, the VM implementation must provide through its UEFI
> > implementation, either:
> > 
> >   a complete FDT which describes the entire VM system and will boot
> >   mainline kernels driven by device tree alone, or
> > 
> >   no FDT.  In this case, the VM implementation must provide ACPI, and
> >   the OS must be able to locate the ACPI root pointer through the UEFI
> >   system table.
> > 
> > For more information about the arm and arm64 boot conventions, see
> > Documentation/arm/Booting and Documentation/arm64/booting.txt in the
> > Linux kernel source tree.
> > 
> > For more information about UEFI and ACPI booting, see [4] and [5].
> 
> What's the point of having ACPI in a virtual machine? You wouldn't
> need to abstract any of the hardware in AML since you already know
> what the virtual hardware is, so I can't see how this would help
> anyone.

The most common response I've been getting so far is that people
generally want their VMs to look close to the real thing, but not sure
how valid an argument that is.

Some people feel strongly about this and seem to think that ARMv8
kernels will only work with ACPI in the future...

Another case is that it's a good development platform.  I know nothing
of developing and testing ACPI, so I won't judge one way or the other.

> 
> However, as ACPI will not be supported by arm32, not having the
> complete FDT will prevent you from running a 32-bit guest on
> a 64-bit hypervisor, which I consider an important use case.
> 

Agreed, I didn't appreciate that fact.  Hmmm, we need to consider that
case.

> > VM Platform
> > -----------
> > The specification does not mandate any specific memory map.  The guest
> > OS must be able to enumerate all processing elements, devices, and
> > memory through HW description data (FDT, ACPI) or a bus-specific
> > mechanism such as PCI.
> > 
> > The virtual platform must support at least one of the following ARM
> > execution states:
> >   (1) aarch32 virtual CPUs on aarch32 physical CPUs
> >   (2) aarch32 virtual CPUs on aarch64 physical CPUs
> >   (3) aarch64 virtual CPUs on aarch64 physical CPUs
> > 
> > It is recommended to support both (2) and (3) on aarch64 capable
> > physical systems.
> 
> Isn't this more of a CPU capabilities question? Or maybe you
> should just add 'if aarch32 mode supported is supported by
> the host CPU'.
> 

The recommendation is to tell people to actually have a -aarch32 (or
whatever it would be called) that works in their VM implementation.
This can certainly be reworded.

> > 
> > The virtual hardware platform must provide a number of mandatory
> > peripherals:
> > 
> >   Serial console:  The platform should provide a console,
> >   based on an emulated pl011, a virtio-console, or a Xen PV console.
> > 
> >   An ARM Generic Interrupt Controller v2 (GICv2) [3] or newer.  GICv2
> >   limits the the number of virtual CPUs to 8 cores, newer GIC versions
> >   removes this limitation.
> > 
> >   The ARM virtual timer and counter should be available to the VM as
> >   per the ARM Generic Timers specification in the ARM ARM [1].
> > 
> >   A hotpluggable bus to support hotplug of at least block and network
> >   devices.  Suitable buses include a virtual PCIe bus and the Xen PV
> >   bus.
> 
> I think you should specify exactly what you want PCIe to look like,
> if present. Otherwise you can get wildly incompatible bus discovery.
> 

As soon as there is more clarity on what it will actually look like,
I'll be happy to add this.  I'm afraid my PCIe understanding is too
piecemeal to fully grasp this, so concrete suggestions for the text
would be much appreciated.

> > Note that this platform specification is separate from the Linux kernel
> > concept of mach-virt, which merely specifies a machine model driven
> > purely from device tree, but does not mandate any peripherals or have any
> > mention of ACPI.
> 
> Did you notice we are removing mach-virt now? Probably no point
> mentioning it here.
> 
Yes, I'm aware.  I've just heard people say "why do we need this, isn't
mach-virt all we need", and therefore I added the note.

I can definitely can rid of this paragraph in the future if it causes
more harm than good.

Thanks!
-Christoffer

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel