|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [PATCH v6 08/44] x86/hvm: introduce per-vCPU L3 page-table
From: Roger Pau Monné <roger.pau@xxxxxxxxxx>
Currently all of a domain's vCPUs share its per-domain area: one L3,
referenced from every root page-table associated with the domain, so
every mapping in it is visible to every vCPU of the domain. Much of the
area is already laid out in per-vCPU slices (each vCPU's COMPAT_ARG_XLAT
pages, and for PV each vCPU's GDT/LDT window), but the visibility is
domain-wide. Meanwhile, much of the per-vCPU state Xen maintains -- the
VMCB, the VMX MSR load/save areas, FPU/XSAVE state -- lives in
always-mapped memory.
For HVM domains, allow each vCPU a per-domain area of its own instead
("vCPU-PT"). HVM monitor tables are already per-vCPU, so this needs
nothing at context switch: every monitor table carries its vCPU's own L3
from creation, and the per-vCPU mappings are isolated from other running
vCPUs at once. We will later build on this, adding a per-vCPU mapcache
and a per-vCPU state window, into which we can move the vCPU state
mentioned above.
Add pervcpu_l3_pg to the arch_vcpu struct, to correspond to the
perdomain_l3_pg in the domain struct. (We retain both so that vCPU-PT
can be chosen domain by domain; PV domains always use the domain-wide
L3.) Modify arch_domain_create() to only allocate a per-domain L3 if
vcpu_pt is not enabled for the domain; modify arch_vcpu_create() to
allocate a per-vCPU L3 if it is.
In {create,destroy}_perdomain_mapping(), if d->arch.asi.vcpu_pt, use
pervcpu_l3_pg as the per-domain L3; otherwise, use perdomain_l3_pg.
Introduce a helper, perdomain_l3(), to consistently choose the correct
one.
Introduce free_pervcpu_mappings() to free this tree, called from
arch_vcpu_destroy() on normal teardown. Since the vcpu structure holds
the only reference to pervcpu_l3_pg, arch_vcpu_create() must also call
it on its error paths: nothing else records the allocation once the
vcpu struct is torn down. The domain-wide free_perdomain_mappings()
keeps covering non-vCPU-PT domains.
init_xen_l4_slots() is given the vCPU that a table is private to.
Have it install that vCPU's perdomain_l3() in slot 260.
Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-4-8, Claude
Code:claude-fable-5-1, Claude Code:claude-opus-5-5
Signed-off-by: George Dunlap <gwd@xxxxxxxxxxxxxx>
---
Changes in v6:
- Port to new explicit L3 allocation regime.
Changes in v5:
- init_xen_l4_slots()'s BUG_ON() follows slot 260 to perdomain_l3().
- Message: all of a domain's vCPUs share its per-domain area; the
mapcache is not among the per-vCPU slices; the mode is "vCPU-PT".
The comment above init_xen_l4_slots() names perdomain_l3(v) and the
domain-wide L3.
Changes in v4:
- Rebase over earlier changes
- d->arch.vcpu_pt is now d->arch.asi.vcpu_pt.
- init_xen_l4_slots() taking a vCPU in place of the domain, and
promote_l4_table() failing while the domain has none, are now patches
of their own (Roger, Jan).
- Drop the fail_early label: every cleanup step after fail: is safe
before its setup has run (Roger).
- free_pervcpu_mappings(): clear the pointer, then free, as XFREE()
does (Roger).
- Likewise in free_perdomain_mappings().
Changes in v3:
- HVM only: PV domains keep the domain-wide area, so the PV parts go
(the single-vCPU limit, the shadow refusals, the XPTI slot install).
init_xen_l4_slots() takes the vCPU as an extra, optional argument
rather than in place of the domain, so PV guest L4s, which no vCPU
owns, are promoted exactly as before.
- Retitle (was: "x86/mm: introduce per-vCPU L3 page-table").
- free_compat_arg_xlat()'s comment names free_pervcpu_mappings() as the
teardown for a vCPU-PT domain.
Changes in v2:
- Added to the series
Changes since the previously posted version:
- Expand the commit message with the motivation and the design
rationale.
- Keep domain-wide freeing intact and introduce a vCPU-scoped
free_pervcpu_mappings() instead of re-scoping
free_perdomain_mappings(); fix the arch_vcpu_create() error-path
leaks of a partially built per-vCPU hierarchy.
- Add a perdomain_l3() helper for the root selection, rather than
open-coding the vcpu_pt choice (and testing both root pointers) at
each site.
---
xen/arch/x86/domain.c | 17 +++++++--
xen/arch/x86/include/asm/domain.h | 4 ++
xen/arch/x86/include/asm/mm.h | 2 +
xen/arch/x86/mm.c | 61 +++++++++++++++++++++++--------
4 files changed, 66 insertions(+), 18 deletions(-)
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 173daf9000..46ca945b48 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -511,16 +511,23 @@ int arch_vcpu_create(struct vcpu *v)
v->arch.flags = TF_kernel_mode;
+ if ( d->arch.asi.vcpu_pt &&
+ (v->arch.pervcpu_l3_pg = allocate_perdomain_l3(d)) == NULL )
+ {
+ rc = -ENOMEM;
+ goto fail;
+ }
+
rc = mapcache_vcpu_init(v);
if ( rc )
- return rc;
+ goto fail;
if ( !is_idle_domain(d) )
{
paging_vcpu_init(v);
if ( (rc = vcpu_init_fpu(v)) != 0 )
- return rc;
+ goto fail;
vmce_init_vcpu(v);
@@ -568,6 +575,7 @@ int arch_vcpu_create(struct vcpu *v)
vcpu_destroy_fpu(v);
xfree(v->arch.msrs);
v->arch.msrs = NULL;
+ free_pervcpu_mappings(v);
return rc;
}
@@ -588,6 +596,8 @@ void arch_vcpu_destroy(struct vcpu *v)
pv_vcpu_destroy(v);
else
ASSERT_UNREACHABLE();
+
+ free_pervcpu_mappings(v);
}
int arch_sanitise_domain_config(const struct xen_domctl_createdomain *config)
@@ -901,7 +911,8 @@ int arch_domain_create(struct domain *d,
psr_domain_init(d);
- if ( (d->arch.perdomain_l3_pg = allocate_perdomain_l3(d)) == NULL )
+ if ( !d->arch.asi.vcpu_pt &&
+ (d->arch.perdomain_l3_pg = allocate_perdomain_l3(d)) == NULL )
{
rc = -ENOMEM;
goto fail;
diff --git a/xen/arch/x86/include/asm/domain.h
b/xen/arch/x86/include/asm/domain.h
index 12f79feb42..4140a92f15 100644
--- a/xen/arch/x86/include/asm/domain.h
+++ b/xen/arch/x86/include/asm/domain.h
@@ -332,6 +332,7 @@ struct monitor_write_data {
struct arch_domain
{
+ /* Domain-wide L3 for the per-domain slot; NULL in a vCPU-PT domain. */
struct page_info *perdomain_l3_pg;
/* I/O-port admin-specified access capabilities. */
@@ -688,6 +689,9 @@ struct arch_vcpu
struct vcpu_msrs *msrs;
+ /* Per-vCPU L3 for the per-domain slot; NULL in a non-vCPU-PT domain. */
+ struct page_info *pervcpu_l3_pg;
+
struct {
bool next_interrupt_enabled;
} monitor;
diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h
index 0d2cecf1c7..7b9a0f561d 100644
--- a/xen/arch/x86/include/asm/mm.h
+++ b/xen/arch/x86/include/asm/mm.h
@@ -369,6 +369,7 @@ int devalidate_page(struct page_info *page, unsigned long
type,
int preemptible);
void init_xen_pae_l2_slots(l2_pgentry_t *l2t, const struct domain *d);
+struct page_info *perdomain_l3(const struct vcpu *v);
void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn,
const struct vcpu *v, mfn_t sl4mfn, bool ro_mpt);
void init_xen_l4_slots_pv(l4_pgentry_t *l4t, mfn_t l4mfn,
@@ -615,6 +616,7 @@ int create_perdomain_mapping(struct vcpu *v, unsigned long
va,
void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va,
unsigned int nr);
void free_perdomain_mappings(struct domain *d);
+void free_pervcpu_mappings(struct vcpu *v);
void __iomem *ioremap_wc(paddr_t pa, size_t len);
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 10106d9f03..f52c72a605 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -1682,6 +1682,18 @@ static int promote_l3_table(struct page_info *page)
}
#endif /* CONFIG_PV */
+/*
+ * The root of the per-domain area in use by @v: the vCPU's own L3 for a
+ * vCPU-PT domain, the domain-wide one otherwise.
+ */
+struct page_info *perdomain_l3(const struct vcpu *v)
+{
+ const struct domain *d = v->domain;
+
+ return d->arch.asi.vcpu_pt ? v->arch.pervcpu_l3_pg
+ : d->arch.perdomain_l3_pg;
+}
+
/*
* Fill an L4 with Xen entries.
*
@@ -1780,8 +1792,11 @@ void init_xen_l4_slots(l4_pgentry_t *l4t, mfn_t l4mfn,
const struct vcpu *v, mfn_t sl4mfn, bool ro_mpt)
{
const struct domain *d = v->domain;
+ struct page_info *l3_pg = perdomain_l3(v);
- _init_xen_l4_slots(l4t, l4mfn, d, d->arch.perdomain_l3_pg, sl4mfn, ro_mpt);
+ BUG_ON(!l3_pg);
+
+ _init_xen_l4_slots(l4t, l4mfn, d, l3_pg, sl4mfn, ro_mpt);
}
/*
@@ -6275,7 +6290,7 @@ int create_perdomain_mapping(struct vcpu *v, unsigned
long va,
struct page_info **ppg)
{
struct domain *d = v->domain;
- struct page_info *pg;
+ struct page_info *pg, *l3_pg = perdomain_l3(v);
l3_pgentry_t *l3tab;
l2_pgentry_t *l2tab;
l1_pgentry_t *l1tab;
@@ -6284,12 +6299,12 @@ int create_perdomain_mapping(struct vcpu *v, unsigned
long va,
ASSERT(va >= PERDOMAIN_VIRT_START &&
va < PERDOMAIN_VIRT_SLOT(PERDOMAIN_SLOTS));
- BUG_ON(!d->arch.perdomain_l3_pg);
+ BUG_ON(!l3_pg);
if ( !nr )
return 0;
- l3tab = __map_domain_page(d->arch.perdomain_l3_pg);
+ l3tab = __map_domain_page(l3_pg);
ASSERT(!l3_table_offset(va ^ (va + nr * PAGE_SIZE - 1)));
@@ -6386,16 +6401,16 @@ void destroy_perdomain_mapping(const struct vcpu *v,
unsigned long va,
unsigned int nr)
{
const l3_pgentry_t *l3tab, *pl3e;
- const struct domain *d = v->domain;
+ struct page_info *l3_pg = perdomain_l3(v);
ASSERT(va >= PERDOMAIN_VIRT_START &&
va < PERDOMAIN_VIRT_SLOT(PERDOMAIN_SLOTS));
ASSERT(!nr || !l3_table_offset(va ^ (va + nr * PAGE_SIZE - 1)));
- if ( !d->arch.perdomain_l3_pg )
+ if ( !l3_pg )
return;
- l3tab = __map_domain_page(d->arch.perdomain_l3_pg);
+ l3tab = __map_domain_page(l3_pg);
pl3e = l3tab + l3_table_offset(va);
if ( l3e_get_flags(*pl3e) & _PAGE_PRESENT )
@@ -6434,16 +6449,11 @@ void destroy_perdomain_mapping(const struct vcpu *v,
unsigned long va,
unmap_domain_page(l3tab);
}
-void free_perdomain_mappings(struct domain *d)
+static void free_perdomain_l3(struct page_info *l3pg)
{
- l3_pgentry_t *l3tab;
+ l3_pgentry_t *l3tab = __map_domain_page(l3pg);
unsigned int i;
- if ( !d->arch.perdomain_l3_pg )
- return;
-
- l3tab = __map_domain_page(d->arch.perdomain_l3_pg);
-
for ( i = 0; i < PERDOMAIN_SLOTS; ++i)
if ( l3e_get_flags(l3tab[i]) & _PAGE_PRESENT )
{
@@ -6479,8 +6489,29 @@ void free_perdomain_mappings(struct domain *d)
}
unmap_domain_page(l3tab);
- free_domheap_page(d->arch.perdomain_l3_pg);
+ free_domheap_page(l3pg);
+}
+
+void free_perdomain_mappings(struct domain *d)
+{
+ struct page_info *l3pg = d->arch.perdomain_l3_pg;
+
+ if ( !l3pg )
+ return;
+
d->arch.perdomain_l3_pg = NULL;
+ free_perdomain_l3(l3pg);
+}
+
+void free_pervcpu_mappings(struct vcpu *v)
+{
+ struct page_info *l3pg = v->arch.pervcpu_l3_pg;
+
+ if ( !l3pg )
+ return;
+
+ v->arch.pervcpu_l3_pg = NULL;
+ free_perdomain_l3(l3pg);
}
static void write_sss_token(unsigned long *ptr)
--
2.56.0
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |