[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH v6 6/6] x86/tlb: use Xen L0 assisted TLB flush when available

  • To: Jan Beulich <jbeulich@xxxxxxxx>
  • From: Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • Date: Thu, 19 Mar 2020 16:40:28 +0100
  • Authentication-results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@xxxxxxxxxx; spf=Pass smtp.mailfrom=roger.pau@xxxxxxxxxx; spf=None smtp.helo=postmaster@xxxxxxxxxxxxxxx
  • Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx, Wei Liu <wl@xxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
  • Delivery-date: Thu, 19 Mar 2020 15:40:55 +0000
  • Ironport-sdr: J1LXgFfnFnBe9C5GbYpGtxLHEyesY91LLYjBwkyyJSTWtijBgA0ln3LcE2ABq+RmijAfc1goyf 5BpjvQNGFLTx49G/Ey123QVWbhYGRLQUqbnNiPu/FX5p2WUHcZPgMsaF/DultTzLKOE8JiAO2u xuKs/nUWOXJbx6hDPwG6kzJlxvUgWVAMidR3qoggAXoQ/+VK74XiCbLoVswFWjZdZ3/dzV4C5q SVR7ntYyDsUuAIGi1/tRUPQtjJrm5YmKjQ7eil+V3PKYoguEeYX20UmoTaU7TKBHf8VyvTgLgi DP0=
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On Fri, Mar 06, 2020 at 10:32:55AM +0100, Jan Beulich wrote:
> On 03.03.2020 18:20, Roger Pau Monne wrote:
> > Use Xen's L0 HVMOP_flush_tlbs hypercall in order to perform flushes.
> > This greatly increases the performance of TLB flushes when running
> > with a high amount of vCPUs as a Xen guest, and is specially important
> > when running in shim mode.
> > 
> > The following figures are from a PV guest running `make -j32 xen` in
> > shim mode with 32 vCPUs and HAP.
> > 
> > Using x2APIC and ALLBUT shorthand:
> > real        4m35.973s
> > user        4m35.110s
> > sys 36m24.117s
> > 
> > Using L0 assisted flush:
> > real    1m2.596s
> > user    4m34.818s
> > sys     5m16.374s
> > 
> > The implementation adds a new hook to hypervisor_ops so other
> > enlightenments can also implement such assisted flush just by filling
> > the hook.
> > 
> > Note that the Xen implementation completely ignores the dirty CPU mask
> > and the linear address passed in, and always performs a global TLB
> > flush on all vCPUs. This is a limitation of the hypercall provided by
> > Xen. Also note that local TLB flushes are not performed using the
> > assisted TLB flush, only remote ones.
> As to this last sentence - isn't this wasteful at least when a
> full address space flush is being processed anyway?

Likely yes, if there are global TLB flushes in parallel to local ones.
AFAICT the purpose of the timestamped TLB is to cope with some of
this, by recording the last TLB stamp and avoiding flushes requested
before that. Here we have to disable such timestamps, since the
flush is done on the back of the vCPU, and hence it's not possible to
accurately record it's timestamp.

From my data above the benefit of using the assisted TLB greatly
outweighs the fact that the timestamped TLB is disabled.

> > Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
> > Reviewed-by: Wei Liu <wl@xxxxxxx>
> Reviewed-by: Jan Beulich <jbeulich@xxxxxxxx>

Thanks, will resend this now.


Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.