[PATCH v4 21/30] xen/xmalloc: return an emptied region after dropping the pool lock

[George Dunlap <gwd@xxxxxxxxxxxxxx>]

When a free leaves a whole region of an xmalloc pool empty,
xmem_pool_free() hands the region back through pool->put_mem() with
pool->lock still held.  For the xenheap pool that is
free_xenheap_pages(), which takes the heap lock inside the pool lock
and runs arch_xenheap_pages_free().  On x86 with a sparse directmap
view configured, the latter takes the page out of the view, at the
cost of one to three rounds of TLB-flush IPIs to every CPU, while every
other CPU's sub-page xmalloc() and xfree() waits on the pool lock.

By then the region is on no free list and no longer counted in the
pool, so nothing else can reach it.  Drop the lock first and return the
region afterwards, as xmem_pool_alloc() already does when it calls
pool->get_mem(), and say at both callbacks that they are called without
the pool lock, so that they keep not relying on it.

Assisted-by: Claude Code:claude-opus-5-5
Signed-off-by: George Dunlap <gwd@xxxxxxxxxxxxxx>
---
 xen/common/xmalloc_tlsf.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/xen/common/xmalloc_tlsf.c b/xen/common/xmalloc_tlsf.c
index b43219b4af..caf33f1a04 100644
--- a/xen/common/xmalloc_tlsf.c
+++ b/xen/common/xmalloc_tlsf.c
@@ -106,7 +106,10 @@ struct xmem_pool {
     unsigned long used_size;
     unsigned long num_regions;
 
-    /* User provided functions for expanding/shrinking pool */
+    /*
+     * User provided functions for expanding/shrinking pool.  They are
+     * called without the pool lock held, and must not rely on it.
+     */
     xmem_pool_get_memory *get_mem;
     xmem_pool_put_memory *put_mem;
 
@@ -488,17 +491,18 @@ void xmem_pool_free(void *ptr, struct xmem_pool *pool)
 
     if ( (b->prev_hdr == NULL) && ((tmp_b->size & BLOCK_SIZE_MASK) == 0) )
     {
-        pool->put_mem(b);
         pool->num_regions--;
         pool->used_size -= BHDR_OVERHEAD; /* sentinel block header */
-        goto out;
+        spin_unlock(&pool->lock);
+        pool->put_mem(b);
+        return;
     }
 
     INSERT_BLOCK(b, pool, fl, sl);
 
     tmp_b->size |= PREV_FREE;
     tmp_b->prev_hdr = b;
- out:
+
     spin_unlock(&pool->lock);
 }
 
-- 
2.55.0