[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [PATCH v4 05/10] xen/domain: Add DOMCTL handler for claiming memory with NUMA awareness


  • To: Teddy Astie <teddy.astie@xxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • From: Bernhard Kaindl <bernhard.kaindl@xxxxxxxxxx>
  • Date: Thu, 26 Feb 2026 23:16:06 +0000
  • Accept-language: en-GB, en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=guVa3+OYmh1pihRKb443wjadAhnEQSn1pbxf4/myDJ4=; b=aaW+vJxRSZOCJ8oimcUEpo1ZyNxpI0sOaGf94a9r+5BZSKmWN8rvzvbnTsI4b71mXfo327SFn8Kj9syuevySahtVLy6rjHq1BZhjYNLQIY0EaUMNDdN6QojmPymlFCmqjh8bpqDXeZBF1h0JY5LKr9zqddAsYTvJYkcPjeKeTdqvd7IEOmooYa0Sh4JyD6pM0iT21AlHU8mGNbRS8oz+oro6PZf3ADLKS343AdBBo9DAZtzXOVa31a5f00zKNOFfHrxhdop0rDFvcReIzkz4tkww8BQ2QeNbCfNfJgafxdxf+rMajssQUsaw+EYE49Qzmpy5tzLTimMGDboWE9aUdw==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=g51g9UsoEcjV5UONX2iZOAvDAKQRYWt7WYgDbYZiG+pX/GXUS6rCw40qY7n2pUZki5cKSo+oRgvA0SfBc/8lFrR2NGq7ayPeOrc48RrEhzjPYk8Zkrvh967FKGFMi94DnMtg28jDJE/oSUoN2zEPkYgggeToVT82E6ciF+1qGYyQaQDnWku4/+Zwfc0/P/P9ozCrCSBtdpa/oXWOiNXOxLWGxHSZGs1KZzkgyniQ1fsCD+zzW67FLIz2EWOCeuDmX/CPRD69ucTxpAmMyfQCf+YqmpQ7NNFOU7huUuV+TyfELdbn2WHrKJmHB6vgB9N8aG9/ZAbhX79ksTPJG960Gw==
  • Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=citrix.com;
  • Cc: Andrew Cooper <andrew.cooper@xxxxxxxxxx>, Anthony PERARD <anthony.perard@xxxxxxxxxx>, Michal Orzel <michal.orzel@xxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>, Julien Grall <julien@xxxxxxx>, Roger Pau Monne <roger.pau@xxxxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, "Daniel P. Smith" <dpsmith@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Thu, 26 Feb 2026 23:16:35 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Thread-index: AQHcpy8NKyz6wh/Wrk6CsdW/rK3ShrWVfM8AgAAgIzA=
  • Thread-topic: [PATCH v4 05/10] xen/domain: Add DOMCTL handler for claiming memory with NUMA awareness

On 26/02/2026 à 22:19, Teddy Astie a écrit :
> Le 26/02/2026 à 15:54, Bernhard Kaindl a écrit :
>> Add a DOMCTL handler for claiming memory with NUMA awareness. It
>> rejects claims when LLC coloring (does not support claims) is enabled
>> and translates the public constant to the internal NUMA_NO_NODE.
>>
>> The request is forwarded to domain_set_outstanding_pages() for the
>> actual claim processing. The handler uses the same XSM hook as the
>> legacy XENMEM_claim_pages hypercall.
>>
>> While the underlying infrastructure currently supports only a single
>> claim, the public hypercall interface is designed to be extensible for
>> multiple claims in the future without breaking the API.
> I'm not sure about the idea of introducing a new hypercall for this
> operation. Though I may be missing some context about the reasons of
> introducing a new hypercall.
>
> XENMEM_claim_pages doesn't have actual support for NUMA, but the
> hypercall interface seems to define it (e.g you can pass
> XENMEMF_exact_node(n) to mem_flags). Would it be preferable instead to
> make XENMEM_claim_pages aware of NUMA-related XENMEMF flags ?

Hello Teddy,

Thank you for your review — much appreciated.

Updating the do_memory_op(XENMEM_claim_pages) handler to accept a node
parameter, as you suggested, is indeed a practical way to retrofit this
feature into existing Xen builds. That’s also the approach we took in
v1 of this series:

* https://lists.xenproject.org/archives/html/xen-devel/2025-03/msg01127.html
* https://patchew.org/Xen/20250314172502.53498-1-alejandro.vallejo@xxxxxxxxx/

We are currently using this approach also in the XS9 Public Preview:

* https://www.xenserver.com/downloads/xs9-preview

That said, during review, Roger Pau Monné suggested that for upstream
inclusion, we should introduce a new hypercall API with support for
multi-node claims, even if the initial infrastructure only handles
a single node. See:

* https://lists.xenproject.org/archives/html/xen-devel/2025-06/msg00484.html

He raised the concern that the current interface effectively constrains
domains to be allocated from one node at a time, or to sequence claims
across nodes, which undermines the purpose of claims.

Instead, he proposed that the hypercall interface would ideally allow
making multi-node claims atomically, rather than requiring multiple
calls with rollback in case of failure.

I favour Roger’s position as well: I think we should aim for a clean
and extensible interface that supports claims across multiple nodes
in a single call. Otherwise, we risk having to introduce yet another
hypercall later when a real-world scenario requires multi-node claims.

On the implementation side, a reliable first-come, first-served mechanism
for multi-node claims will require serialisation in the central claim path.
Currently, the global heap_lock provides that protection, and it would
naturally cover the creation of a multi-node claim under a single lock,
ensuring atomicity and consistent behaviour.

Thanks again for the review and feedback!

Best regards / Bien cordialement / Saludos / Liebe Grüße,

With warm greetings from Vienna/Austria,
Bernhard

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.