[RFC PATCH 0/7] x86: Use a single fixed ASID per domain

["Teddy Astie" <teddy.astie@xxxxxxxxxx>]

(ASID can be replaced with VPID for Intel)

This patch series is a reorganized version of [1] though, it share the same 
goal.
Not a lot of logic remain from v1/v2 [2] and the patch series is structured 
quite
differently to v3 [1].

Currently in Xen. In order to perform a TLB flush for a guest, we just increment
the ASID, which has the same effect as flushing the TLB. When we run out of 
ASID,
we flush the entire TLB and start over.
This is done per pCPU, thus the ASID management is done on a per-pCPU basis.

However, AMD SEV requires the ASID to be configured in advance and be unique per
domain as it bound to the encryption key used for the domain.

This patch series propose a new ASID model where a fixed single ASID is 
attributed
to the domain, thus all vCPU share such ASID. Moreover, the TLB for a vCPU is 
always
flushed in these conditions :
* the vCPU is migrated to another pCPU
(remote pCPU TLB may be out of sync or lack flushes made previously)
* a different vCPU of the same domain previously ran on this pCPU
(TLB entries comes from the previous vCPU, which may have a different CR3)

Albeit related to a SEV vulnerability, [4] gives more justifications on the 
needs
of these rules for correctness.

It is also important for making hypervisor-side guest broadcast TLB flushing
realistic, as a single fixed ASID per domain allows a broacast TLB flush to use
such ASID as target.

Moreover, AMD programmer manual also explicitely require this for correctly 
virtualizing
INVLPGB in guests (through 090h:7 ("Enable INVLPGB/TLBSYNC.") in VMCB) :
> A guest that executes a legal INVLPGB that is not intercepted will have the 
> requested
> ASID field replaced by the current ASID and the valid ASID bit set before 
> doing the
> broadcast invalidation.  Because of its broadcast nature, the ASID field must 
> be global
> and all processors must allocate the same ASID to the same Guest for proper 
> operation.
> Hypervisors that do not support a global ASID must intercept the Guest usage 
> of INVLPGB,
> if enabled, for proper behavior.

In order to avoid making ASID management too complex, transition to only using 
a fixed ASID
per domain. If no ASID is available (e.g out of ASID or ASID usage is diabled), 
we use ASID=1
as a placeholder (which is still a valid ASID), and always flush the TLB on 
context switch
if the vcpu is using ASID=1.

Also add a way to specify a prefered minimum ASID, it is meant to be used later 
on to avoid
clobbering the low ASID space of SEV-ES and favor allocating ASIDs over that 
space.

[1] x86/hvm: Introduce Xen-wide ASID allocator
https://lore.kernel.org/xen-devel/cover.1750770621.git.teddy.astie@xxxxxxxxxx/

[2] x86/hvm: Introduce Xen-wide ASID allocator
https://lore.kernel.org/xen-devel/cover.1723574652.git.vaishali.thakkar@xxxxxxxxxx/

[4] TLB Poisoning Attacks on AMD Secure Encrypted Virtualization
https://dl.acm.org/doi/epdf/10.1145/3485832.3485876

Teddy Astie (7):
  vmx: Introduce vcpu single context VPID invalidation
  common: Track latest pCPU that ran the vCPU
  common: Introduce needs_tlb_flush vcpu field
  x86: Set v->needs_tlb_flush when needed
  x86/hvm: Flush TLB on vCPU overlaps on the same pCPU
  x86/hvm: Transition to needs_tlb_flush logic, use per-domain ASID
  hvm: Allow specifying a prefered asid minimum

 docs/misc/xen-command-line.pandoc      |   2 +-
 xen/arch/x86/flushtlb.c                |  22 +--
 xen/arch/x86/hvm/asid.c                | 177 +++++++++++--------------
 xen/arch/x86/hvm/emulate.c             |   2 +-
 xen/arch/x86/hvm/hvm.c                 |  25 +++-
 xen/arch/x86/hvm/nestedhvm.c           |   7 +-
 xen/arch/x86/hvm/svm/asid.c            |  67 +++++++---
 xen/arch/x86/hvm/svm/nestedsvm.c       |   2 +-
 xen/arch/x86/hvm/svm/svm.c             |  47 +++++--
 xen/arch/x86/hvm/svm/svm.h             |   4 -
 xen/arch/x86/hvm/vmx/vmcs.c            |   6 +-
 xen/arch/x86/hvm/vmx/vmx.c             |  77 ++++++-----
 xen/arch/x86/hvm/vmx/vvmx.c            |   4 +-
 xen/arch/x86/include/asm/flushtlb.h    |   7 -
 xen/arch/x86/include/asm/hvm/asid.h    |  27 ++--
 xen/arch/x86/include/asm/hvm/domain.h  |   1 +
 xen/arch/x86/include/asm/hvm/hvm.h     |  15 +--
 xen/arch/x86/include/asm/hvm/svm.h     |   5 +
 xen/arch/x86/include/asm/hvm/vcpu.h    |  10 +-
 xen/arch/x86/include/asm/hvm/vmx/vmx.h |  23 +++-
 xen/arch/x86/mm/hap/hap.c              |   9 +-
 xen/arch/x86/mm/p2m.c                  |   7 +-
 xen/arch/x86/mm/paging.c               |   2 +-
 xen/arch/x86/mm/shadow/multi.c         |  12 +-
 xen/common/domain.c                    |   8 ++
 xen/common/sched/core.c                |   5 +
 xen/include/xen/sched.h                |   6 +
 27 files changed, 319 insertions(+), 260 deletions(-)

-- 
2.52.0



--
Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech