[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH] xen/dom0less: Increase guest DTB size for high-vCPU guests


  • To: Oleksandr Tyshchenko <Oleksandr_Tyshchenko@xxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • From: Grygorii Strashko <grygorii_strashko@xxxxxxxx>
  • Date: Tue, 2 Dec 2025 23:33:34 +0200
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w3imEDtJhDTR4KPhpBxwU3oT7pPoKK5KcXrl6UfJt+M=; b=j3n3q2qO8049q1OwzjEDMYfjRhJ1Q+ACpyjCRA4ORT9DAUIm3kmFheDiQfirQ7SckOWDbs3b/p2U7Gz/I6m2pOH05Sys/nGVbYuK7HT/kt370ff/SOMcKslgYcUiRLhRygYL6A3apHh5wQh5gFKwau9La7oonoVY4XZeTRPcTP+qaPYKcQfznl1nrkQS/FMKnWyPjNbu689H4135dbINVru8FuX+iH3JroXmCRgBtMGk0zYwqDSUdqgOQFJ+Sl40yb2VidqgtXCewSRgTu7B/JaTkuPm/jONspnlH4EMAkC+Id+CYcM5s7kb8chYWEHLuv/Pdri4mR9UjOzimC37Bg==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NhWuLEoAh1Hz29yNuPV1Ti8mjyNvQPTNIK668211/Jpo8qY5ChyoySPUyrwa1khgjVUTHj2XZ1cxJBRA2XOyj7RGd9YGgjqtVf7LBoSRB75WDqzpdNDGEGTfDMrrixLiYF8y1i2TfEmSxiMPZW3Qxm22j11z0eZRVNjj2hhoUs5MnanFgtpBMM3y1URjO07H60VOTVbSrIhpzB2gViMJy9cjlE1pkmq/yJLzbXnH5tp8CepM4MUX4IvuNs2YEkRIIXSNfA+bJzJts2RoHGrwgxPuB1kbVytfpG+K3ix9acnVIi8ohcu0mci/wjX1rnUULRD12B8Hl96ENruiO3QkYg==
  • Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=epam.com;
  • Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>, Julien Grall <julien@xxxxxxx>, Bertrand Marquis <bertrand.marquis@xxxxxxx>, Michal Orzel <michal.orzel@xxxxxxx>
  • Delivery-date: Tue, 02 Dec 2025 21:34:13 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>



On 02.12.25 21:32, Oleksandr Tyshchenko wrote:
Creating a guest with a high vCPU count (e.g., >32) fails because
the guest's device tree buffer (DOMU_DTB_SIZE) overflows during creation.
The FDT nodes for each vCPU quickly exhaust the 4KiB buffer,
causing a guest creation failure.

Increase the buffer size to 16KiB to support guests up to
the MAX_VIRT_CPUS limit (128).

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@xxxxxxxx>
---
Noticed when testing the boundary conditions for dom0less guest
creation on Arm64.

Domain configuration:
fdt mknod /chosen domU0
fdt set /chosen/domU0 compatible "xen,domain"
fdt set /chosen/domU0 \#address-cells <0x2>
fdt set /chosen/domU0 \#size-cells <0x2>
fdt set /chosen/domU0 memory <0x0 0x10000 >
fdt set /chosen/domU0 cpus <33>
fdt set /chosen/domU0 vpl011
fdt mknod /chosen/domU0 module@40400000
fdt set /chosen/domU0/module@40400000 compatible  "multiboot,kernel" 
"multiboot,module"
fdt set /chosen/domU0/module@40400000 reg <0x0 0x40400000 0x0 0x16000 >
fdt set /chosen/domU0/module@40400000 bootargs "console=ttyAMA0"

Failure log:
(XEN) Xen dom0less mode detected
(XEN) *** LOADING DOMU cpus=33 memory=0x10000KB ***
(XEN) Loading d1 kernel from boot module @ 0000000040400000
(XEN) Allocating mappings totalling 64MB for d1:
(XEN) d1 BANK[0] 0x00000040000000-0x00000044000000 (64MB)
(XEN) Device tree generation failed (-22).
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Could not set up domain domU0 (rc = -22)
(XEN) ****************************************
---
---
  xen/common/device-tree/dom0less-build.c | 8 +++++---
  1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/xen/common/device-tree/dom0less-build.c 
b/xen/common/device-tree/dom0less-build.c
index 3f5b987ed8..d7d0a47b97 100644
--- a/xen/common/device-tree/dom0less-build.c
+++ b/xen/common/device-tree/dom0less-build.c
@@ -461,10 +461,12 @@ static int __init domain_handle_dtb_boot_module(struct 
domain *d,
/*
   * The max size for DT is 2MB. However, the generated DT is small (not 
including
- * domU passthrough DT nodes whose size we account separately), 4KB are enough
- * for now, but we might have to increase it in the future.
+ * domU passthrough DT nodes whose size we account separately). The size is
+ * primarily driven by the number of vCPU nodes. The previous 4KiB buffer was
+ * insufficient for guests with high vCPU counts, so it has been increased
+ * to support up to the MAX_VIRT_CPUS limit (128).
   */
-#define DOMU_DTB_SIZE 4096
+#define DOMU_DTB_SIZE (4096 * 4)

May be It wants Kconfig?
Or some formula which accounts MAX_VIRT_CPUS?

  static int __init prepare_dtb_domU(struct domain *d, struct kernel_info 
*kinfo)
  {
      int addrcells, sizecells;

--
Best regards,
-grygorii




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.