Re: [RFC PATCH] xen/memory: Introduce a hypercall to provide unallocated space

[Julien Grall <julien@xxxxxxx>]

Hi Jan,

On 03/08/2021 13:49, Jan Beulich wrote:
> On 28.07.2021 21:53, Julien Grall wrote:
> > On 28/07/2021 20:00, Andrew Cooper wrote:
> > > On 28/07/2021 18:27, Julien Grall wrote:
> > > > On 28/07/2021 18:19, Andrew Cooper wrote:
> > > > > On 28/07/2021 17:18, Oleksandr Tyshchenko wrote:
> > > > > > From: Oleksandr Tyshchenko <oleksandr_tyshchenko@xxxxxxxx>
> > > > > >
> > > > > > Add XENMEM_get_unallocated_space hypercall which purpose is to
> > > > > > query hypervisor to find regions of guest physical address space
> > > > > > which are unused and can be used to create grant/foreign mappings
> > > > > > instead of wasting real pages from the domain memory for
> > > > > > establishing these mappings. The problem with the current Linux
> > > > > > on Xen on Arm behaviour is if we want to map some guest memory
> > > > > > regions in advance or to perform cache mappings in the backend
> > > > > > we might run out of memory in the host (see XSA-300).
> > > > > > This of course, depends on the both host and guest memory sizes.
> > > > > >
> > > > > > The "unallocated space" can't be figured out precisely by
> > > > > > the domain on Arm without hypervisor involvement:
> > > > > > - not all device I/O regions are known by the time domain starts
> > > > > >      creating grant/foreign mappings
> > > > > > - the Dom0 is not aware of memory regions used for the identity
> > > > > >      mappings needed for the PV drivers to work
> > > > > > In both cases we might end up re-using these regions by
> > > > > > a mistake. So, the hypervisor which maintains the P2M for the domain
> > > > > > is in the best position to provide "unallocated space".
> > > > > I'm afraid this does not improve the situation.
> > > > >
> > > > > If a guest follows the advice from XENMEM_get_unallocated_space, and
> > > > > subsequently a new IO or identity region appears, everything will
> > > > > explode, because the "safe area" wasn't actually safe.
> > > > >
> > > > > The safe range *must* be chosen by the toolstack, because nothing else
> > > > > can do it safely or correctly.
> > > > The problem is how do you size it? In particular, a backend may map
> > > > multiple time the same page (for instance if the page is granted twice).
> > > The number of mapped grants is limited by the size of the maptrack table
> > > in Xen, which is a toolstack input to the domaincreate hypercall.
> > > Therefore, the amount of space required is known and bounded.
> > >
> > > There are a handful of other frames required in the current ABI (shared
> > > info, vcpu info, etc).
> > >
> > > The areas where things do become fuzzy is things like foreign mappings,
> > > acquire_resource, etc for the control domain, which are effectively
> > > unbounded from the domain's point of view.
> > >
> > > For those, its entirely fine to say "here 128G of safe mapping space" or
> > > so.  Even the quantity of mapping dom0 can make is limited by the shadow
> > > memory pool and the number of pagetables Xen is willing to expend on the
> > > second stage translation tables.
> > FWIW, on Arm, we don't have shadow memory pool.
> Where do you take 2nd level page table memory from then?
From xenheap directly. And yes, I know this may lead to other issues. 
But that's a known missing features that so far no-one tackled.
Cheers,

--
Julien Grall