Re: [Xen-devel] Future support of 5-level paging in Xen:wq

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Mon, 12 Dec 2016, Juergen Gross wrote:
> On 09/12/16 20:31, Stefano Stabellini wrote:
> > On Fri, 9 Dec 2016, Juergen Gross wrote:
> >> On 09/12/16 00:50, Stefano Stabellini wrote:
> >>> On Thu, 8 Dec 2016, Andrew Cooper wrote:
> >>>> On 08/12/2016 19:18, Stefano Stabellini wrote:
> >>>>> On Thu, 8 Dec 2016, Andrew Cooper wrote:
> >>>>>> On 08/12/16 16:46, Juergen Gross wrote:
> >>>>>>> The first round of (very preliminary) patches for supporting the new
> >>>>>>> 5-level paging of future Intel x86 processors [1] has been posted to
> >>>>>>> lkml:
> >>>>>>>
> >>>>>>> https://lkml.org/lkml/2016/12/8/378
> >>>>>>>
> >>>>>>> An explicit note has been added: "CONFIG_XEN is broken." and
> >>>>>>> "I would appreciate help with the code."
> >>>>>>>
> >>>>>>> I think we should start a discussion what we want to do in future:
> >>>>>>>
> >>>>>>> - are we going to support 5-level paging for PV guests?
> >>>>>>> - do we limit 5-level paging to PVH and HVM?
> >>>>>> The 64bit PV ABI has 16TB of virtual address space just above the upper
> >>>>>> 48-canonical boundary.
> >>>>>>
> >>>>>> Were Xen to support 5-level PV guests, we'd either leave the PV guest
> >>>>>> kernel with exactly the same amount of higher half space as it 
> >>>>>> currently
> >>>>>> has, or we'd have to recompile Xen as properly position-independent and
> >>>>>> use a different virtual range in different paging mode.
> >>>>>>
> >>>>>> Another pain point is the quantity of virtual address space handed away
> >>>>>> in the ABI.  We currently had 97% of the virtual address space away to
> >>>>>> 64bit PV guests, and frankly this is too much.  This is the wrong way
> >>>>>> around when Xen has more management to do than the guest.  If we were 
> >>>>>> to
> >>>>>> go along the 5-level PV guests route, I will insist that there is a
> >>>>>> rather more even split of virtual address space baked into the ABI.
> >>>>>>
> >>>>>> However, a big question is whether any of this effort is worth doing, 
> >>>>>> in
> >>>>>> the light of PVH.
> >>>>> With my Aporeto hat on, I'll say that given the overwhelming amount of
> >>>>> hardware available out there without virtualization support, this work
> >>>>> is worth doing. I am referring to all the public cloud virtual machines,
> >>>>> which can support Xen PV guests but cannot support PVH guests.
> >>>>
> >>>> Why is Xen supporting 5-level guests useful for running in a PV cloud
> >>>> VM?  Xen doesn't run PV.
> >>>>
> >>>> I am not suggesting that we avoid adding 5-level support to Xen.  We
> >>>> should absolutely do that.  The question is only whether we extend the
> >>>> PV ABI to support 5-level PV guests.  Conceptually, its very easy to
> >>>> have a 5-level Xen but only supporting 4-level PV guests.
> >>>>
> >>>> VT-x and SVM date from 2005/2006 and are now 10 years old.  I would be
> >>>> surprised if you would find much hardware of this age in any cloud; you
> >>>> can't by anything that old, and support contracts have probably run out
> >>>> if you have owned that hardware for 10 years.
> >>>
> >>> I am thinking that in a couple of years, we might already find VMs so
> >>> large that to use all the memory in a nested virt scenario, we need
> >>> 5-level PV abi support.
> >>>
> >>
> >> No, I don't think so. I believe there will be no hardware capable of
> >> 5-level paging but without VMX/SVM support. Support of PVH/HVM for such
> >> large guests should be enough. We don't need to extend PV which we want
> >> to get rid of in Linux anyway, no?
> > 
> > I am talking about nested virtualization when the L1 virtual machine
> > does not support nested VMX or SVM. No Amazon AWS virtual machines
> > support nested VMX, but it is possible to run Xen PV virtual machines on
> > top of any Amazon HVM instance.
> > 
> > When 5-level pagetable hardware will become available on Amazon AWS, it
> > might be possible to get virtual machines so large that in order to use
> > all the memory, you need to use 5-level pagetables in L1 Xen. In this
> > scenario, if we want to create a L2 virtual machine as large as possible
> > we will need support for 5-level page tables in the PV ABI.
> > 
> > Please correct me if I am wrong.
> 
> So basically you are telling us that due to nested virt on AWS we will
> never be able to drop PV support, right?

That's right, although never say never :-)


> Why are we pushing PVH then?

Because it is significantly faster than PV on most workloads on real
hardware, while retaining the security properties of PV. Definitely a
vital option to have on platforms that have VMX or SVM.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel