|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [PATCH v7 02/50] x86/mm: introduce a per-vCPU state window in the perdomain area
Guest-owned register state that Xen keeps in always-mapped memory, on
the xenheap or in the xmalloc pool, is to move to the domain heap,
reached through mappings of its own rather than through the directmap.
The long-lived per-vCPU structures involved (the SVM VMCB, the VMX
guest MSR load/save list, the guest XSAVE area, and later the
aggregated small per-vCPU allocations) all want the same shape: one or
more domain-heap pages, mapped once at fixed per-vCPU virtual addresses
for the life of the vCPU.
Add a fourth per-domain-mapping sub-area providing that destination: a
small per-vCPU stride of page slots, with a fixed index assigned to
each relocated structure and the page tables created at vCPU creation
time. Consumers allocate their own pages and install them with
populate_perdomain_mapping(); the backing pages remain caller-owned,
so their lifetime stays with the consumer, while the page tables are
torn down with the rest of the per-domain area.
Before freeing their pages, consumers take them out of the window with
unmap_vcpu_state(), on failure paths as at teardown: vCPU creation can
fail once a consumer has mapped its pages, and the domain lives on.
It flushes the domain's dirty CPUs: the window is in the per-domain
area that all of the domain's vCPUs share, so these are the CPUs whose
page-tables carry it. So far that mask is empty on every call, as the
vCPU has not run or the domain is paused. A consumer reaching a page
from another context, through a transient map_domain_page() mapping,
drops the mapping with unmap_vcpu_state_page(), which also flushes this
CPU's translations of it, there and at its alias in the per-domain
mirror: the mapcache defers that flush, and the translations would
otherwise outlive the mapping.
The window has the same domain-wide visibility as the other
per-domain sub-areas: each vCPU's stride, at virtual addresses keyed on
vcpu_id, is mapped in the page tables of all of the domain's vCPUs.
This is infrastructure for moving the SVM VMCB, the VMX MSR load/save
lists, the saved register frames, the HVM emulator's caches and the
guest XSAVE areas off the xenheap. Slots specific to AMD or Intel can
share the same space. Nothing occupies the area yet; the only
behavioural change is the page-table allocation itself and the failure
path that comes with it: vCPU creation can now fail with -ENOMEM.
Both apply to every guest vCPU, PV ones included. A domain gets one
L2 plus an L1 per 64 vCPUs: 2 pages for up to 64 vCPUs. A PV domain's
window stays empty until the register frames and XSAVE areas move
there.
Assisted-by: Claude Code:claude-fable-5, Claude Code:claude-opus-5-5
Signed-off-by: George Dunlap <gwd@xxxxxxxxxxxxxx>
---
Changes in v7:
- Standalone on staging, without per-vCPU page tables or the sparse
directmap view: create_perdomain_mapping() is given the domain, and
the comments and the message describe the domain-wide area and
staging's mapcache alone. The window's comment in config.h says its
structures are kept on the domain heap, not out of always-mapped
memory.
- Have AMD- and Intel-specific sections that share the same slots (Jan Beulich)
Changes in v6:
- Port over change to L3 allocation.
Changes in v5:
- Say that the PV register frames move into the window too, not only
the XSAVE areas.
- Message and comments: "per-domain area", and which vCPUs share it.
Changes in v4:
- New in this version.
---
xen/arch/x86/domain.c | 12 +++++++++
xen/arch/x86/include/asm/config.h | 23 ++++++++++++++++-
xen/arch/x86/include/asm/mm.h | 2 ++
xen/arch/x86/mm.c | 42 +++++++++++++++++++++++++++++++
4 files changed, 78 insertions(+), 1 deletion(-)
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 74513a79a7..2407c575d9 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -517,6 +517,18 @@ int arch_vcpu_create(struct vcpu *v)
if ( !is_idle_domain(d) )
{
+ BUILD_BUG_ON((MAX_VIRT_CPUS << VCPU_STATE_VA_SHIFT) >
+ (PERDOMAIN_SLOT_MBYTES << 20));
+ BUILD_BUG_ON(VCPU_STATE_IDX_VMCB >= VCPU_STATE_NR_PAGES);
+ BUILD_BUG_ON(VCPU_STATE_IDX_N2VMCB >= VCPU_STATE_NR_PAGES);
+ BUILD_BUG_ON(VCPU_STATE_IDX_VMX_MSR >= VCPU_STATE_NR_PAGES);
+ /* Page tables only; torn down with the rest of the per-domain area. */
+ rc = create_perdomain_mapping(d, VCPU_STATE_START(v),
+ VCPU_STATE_NR_PAGES,
+ NIL(l1_pgentry_t *), NULL);
+ if ( rc != 0 )
+ goto fail;
+
paging_vcpu_init(v);
if ( (rc = vcpu_init_fpu(v)) != 0 )
diff --git a/xen/arch/x86/include/asm/config.h
b/xen/arch/x86/include/asm/config.h
index aebd3e8f45..b05dbd4a4b 100644
--- a/xen/arch/x86/include/asm/config.h
+++ b/xen/arch/x86/include/asm/config.h
@@ -158,7 +158,7 @@
/* Slot 260: per-domain mappings (including map cache). */
#define PERDOMAIN_VIRT_START (PML4_ADDR(260))
#define PERDOMAIN_SLOT_MBYTES (PML4_ENTRY_BYTES >> (20 + PAGETABLE_ORDER))
-#define PERDOMAIN_SLOTS 3
+#define PERDOMAIN_SLOTS 4
#define PERDOMAIN_VIRT_SLOT(s) (PERDOMAIN_VIRT_START + (s) * \
(PERDOMAIN_SLOT_MBYTES << 20))
/* Slot 4: mirror of per-domain mappings (for compat xlat area accesses). */
@@ -291,6 +291,27 @@ extern unsigned long xen_phys_start;
#define ARG_XLAT_START(v) \
(ARG_XLAT_VIRT_START + ((v)->vcpu_id << ARG_XLAT_VA_SHIFT))
+/*
+ * Per-vCPU state window. The fourth per-domain-mapping sub-area: a small
+ * per-vCPU stride of page slots holding long-lived per-vCPU structures kept
+ * on the domain heap, each mapped once for the life of the vCPU at a fixed
+ * index within the stride.
+ */
+#define VCPU_STATE_SLOT 3
+#define VCPU_STATE_VIRT_START PERDOMAIN_VIRT_SLOT(VCPU_STATE_SLOT)
+#define VCPU_STATE_ORDER 3
+#define VCPU_STATE_NR_PAGES (1U << VCPU_STATE_ORDER)
+#define VCPU_STATE_VA_SHIFT (VCPU_STATE_ORDER + PAGE_SHIFT)
+#define VCPU_STATE_START(v) \
+ (VCPU_STATE_VIRT_START + ((v)->vcpu_id << VCPU_STATE_VA_SHIFT))
+#define VCPU_STATE_ADDR(v, idx) \
+ (VCPU_STATE_START(v) + (idx) * PAGE_SIZE)
+/* AMD-specific slots */
+#define VCPU_STATE_IDX_VMCB 0
+#define VCPU_STATE_IDX_N2VMCB 1
+/* Intel-specific slots */
+#define VCPU_STATE_IDX_VMX_MSR 0
+
#define ELFSIZE 64
#define ARCH_CRASH_SAVE_VMCOREINFO
diff --git a/xen/arch/x86/include/asm/mm.h b/xen/arch/x86/include/asm/mm.h
index 5de04b32aa..bbce8c401e 100644
--- a/xen/arch/x86/include/asm/mm.h
+++ b/xen/arch/x86/include/asm/mm.h
@@ -615,6 +615,8 @@ void populate_perdomain_mapping(const struct vcpu *v,
unsigned long va,
unsigned int flags);
void destroy_perdomain_mapping(const struct vcpu *v, unsigned long va,
unsigned int nr);
+void unmap_vcpu_state(const struct vcpu *v, unsigned int idx, unsigned int nr);
+void unmap_vcpu_state_page(const void *va);
void free_perdomain_mappings(struct domain *d);
void __iomem *ioremap_wc(paddr_t pa, size_t len);
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 9e313e2a82..7cfeceb8c6 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -6470,6 +6470,48 @@ void destroy_perdomain_mapping(const struct vcpu *v,
unsigned long va,
unmap_domain_page(l3tab);
}
+/*
+ * Take @nr slots of @v's state window, from @idx, out of the per-domain
+ * area, before the caller frees the pages behind them: the pages are the
+ * caller's own, and must not stay mapped once freed. The window's
+ * translations are usable only on CPUs whose page-tables carry it: those of
+ * any of the domain's vCPUs, all in the domain's dirty mask.
+ */
+void unmap_vcpu_state(const struct vcpu *v, unsigned int idx, unsigned int nr)
+{
+ const struct domain *d = v->domain;
+
+ /* Beyond the stride is the next vCPU's window. */
+ BUG_ON(idx >= VCPU_STATE_NR_PAGES || nr > VCPU_STATE_NR_PAGES - idx);
+
+ destroy_perdomain_mapping(v, VCPU_STATE_ADDR(v, idx), nr);
+
+ flush_tlb_mask(d->dirty_cpumask);
+}
+
+/*
+ * Drop a transient mapping of a page behind a vCPU's state window, and this
+ * CPU's translations of it: at its address, and at its alias in the
+ * per-domain mirror, which every domain but a 64-bit PV one has (flushing
+ * that address there is harmless). The mapcache does not drop them with
+ * the slot's last mapping: it defers the flush, and CPUs running other vCPUs
+ * of its domain may have cached the slot too. A directmap address, where
+ * the caller has no mapcache, needs nothing.
+ */
+void unmap_vcpu_state_page(const void *va)
+{
+ unsigned long addr = (unsigned long)va;
+
+ unmap_domain_page(va);
+
+ if ( addr < DIRECTMAP_VIRT_START )
+ {
+ flush_tlb_one_local(addr);
+ flush_tlb_one_local(addr - PERDOMAIN_VIRT_START +
+ PERDOMAIN_ALT_VIRT_START);
+ }
+}
+
void free_perdomain_mappings(struct domain *d)
{
l3_pgentry_t *l3tab;
--
2.56.0
|
![]() |
Lists.xenproject.org is hosted with RackSpace, monitoring our |