Re: [PATCH v3 15/18] x86/mm: flush owned pages before reuse with the sparse directmap view

[George Dunlap <gwd@xxxxxxxxxxxxxx>]

On Thu, Oct 15, 2026 at 5:46 PM Roger Pau Monné <roger@xxxxxxxxxxxxxx> wrote:
>
> On Wed, Oct 14, 2026 at 04:34:54PM +0100, George Dunlap wrote:
> > On Thu, Oct 8, 2026 at 8:00 PM Roger Pau Monné <roger@xxxxxxxxxxxxxx> wrote:
> > >
> > > On Wed, Oct 07, 2026 at 11:40:48AM +0100, George Dunlap wrote:
> > > > Since d7bf1b8ea275 ("x86/mm: limit deferred TLB flushing to PV domain
> > > > support") a page freed by its owner is flushed from TLBs before it is
> > > > handed out again only in builds with PV support, on the premise that PV
> > > > domains, with their (limited) control over the host MMU and over when
> > > > flushes happen, are the only ones to leave stale entries behind.
> > > >
> > > > Xen's transient mappings of domain pages are another source.  The
> > > > mapcache zaps a slot when it is unmapped but defers the flush: the
> > > > slot's translation can stay in the TLB of the CPU which used it until
> > > > that CPU next flushes, by which time the page may have been freed and
> > > > handed to another domain.  Before "x86/hvm: introduce per-vCPU
> > > > mapcache for vCPU-PT domains", only PV vCPUs use the mapcache, so the
> > > > PV condition covers it.  Now that the vCPUs of an HVM domain with
> > > > per-vCPU page-tables have one as well.
> > > >
> > > > While the full directmap is loaded in every context, stale transient
> > > > mappings only map what's already mapped in the full directmap.  On the
> > > > sparse view of the directmap, a stale translation of a page handed to
> > > > another domain would keep it reachable from a context which has
> > > > nothing to do with it.
> > >
> > > But if it's handed to another domain, then it also hasn't gone through
> > > a free cycle, which is what pg->u.free.need_tlbflush covers?
> >
> > Sorry, the wording there is a bit imprecise; it should be "a stale
> > translation of a page *freed and then* handed to another domain".
> >
> > Apart from things like steal_page(), which doesn't apply to HVM guests
> > AFAIK, there isn't any way to hand a page to another domain, is there?
> >
> > I guess there's dom_cow for memory sharing; but that's experimental.
> > We should just document that ASI + memory sharing isn't supported at
> > the moment, and work out the details when that's wanted.
> >
> > > > So flush owned pages before reuse wherever a sparse view is
> > > > configured, as before d7bf1b8ea275.  In builds without PV support,
> > > > mark_page_free() now also asks a new hook,
> > > > arch_freed_pages_need_tlbflush(), declared with the xenheap hooks the
> > > > sparse view already uses.  x86 answers with sparse_dmap_configured();
> > > > without CONFIG_SPARSE_DIRECTMAP, and on other architectures, the
> > > > answer is no.  Builds with PV support flush as before, and HVM-only
> > > > builds keep d7bf1b8ea275's saving unless a sparse view is configured,
> > > > which nothing does until a later patch.
> > >
> > > I think this is not fully correct, in fact I think we need to flush
> > > the map cache on every return to guest, otherwise you risk leaking
> > > contents between processes running on the same vCPU (and by extension
> > > the same physical CPU).
> > >
> > > We either need a flush flush before returning to guest, or we need to
> > > invalidate every page that unmapped from the mapcache.
> >
> > This is a good point, but it's raising the bar significantly in terms
> > of what kinds of attacks we're trying to defend against.  Stack rubble
> > could technically contain sensitive guest data, but is much harder to
> > take advantage of than the full direct map, and XSAVE registers in the
> > xenheap; so it's delayed until later.  I think this is the same kind
> > of thing: Yes, pages mapped by a hypercall may contain sensitive data
> > relevant to other processes, but it's kind of like stack rubble: much
> > less useful.
> >
> > My recommendation is that we put this off, past moving XSAVE out of
> > the xenheap, and treat it as another level of safety.
>
> Up to a degree there's also a performance aspect here: when you set
> pg->u.free.need_tlbflush that triggers a host-wide flush on the next
> allocation (if needed, which is likely the case for systems running
> HVM-only domains).  However if we do the flush on mapcahce unmap,
> that can be a single invlpg on the local CPU.  As systems grow larger,
> a host-wide TLB flush is quite a bottleneck, and it also introduces
> interference to remote CPUs.
>
> I would rather not re-enable the usage of pg->u.free.need_tlbflush, as
> I think for a complete ASI solution we will need to solve this
> flushing TLB entries when pages are unmapped from the mapcache.  I
> would possibly leave this with a TODO or comment somewhere.

Well leaving this undone leaves potential cross-domain data windows open.

You're right, avoiding IPIs on free, is a big win; the cost is the
INVLPG, and a TLB re-walk on repeated references.

I'll give it a think, but probably go with your suggestion.

 -George