Xen project Mailing List

Hi Jan, thanks for the response! Adding more details and log files here:

1. I installed Xen 4.14 on CentOS 7 with yum:

[root@test-xen ~]# yum list xen kernel
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: download.cf.centos.org
* centos-virt-xen-epel: d2lzkl7pfhq30w.cloudfront.net
* extras: download.cf.centos.org
* updates: download.cf.centos.org
Installed Packages
kernel.x86_64 3.10.0-1160.45.1.el7 installed
kernel.x86_64 3.10.0-1160.76.1.el7 @updates
kernel.x86_64 4.9.241-37.el7 @centos-virt-xen-414
xen.x86_64 4.14.5.24.g87d90d511c-1.el7 @centos-virt-xen-414
[root@test-xen ~]# uname -r

4.9.241-37.el7.x86_64

2. I configured dom0_max_vcpus=16 for Xen, and nr_cpus=16 for kernel:

[root@test-xen ~]# xl info
host : ip-172-31-52-150.us-west-2.compute.internal
release : 4.9.241-37.el7.x86_64
version : #1 SMP Mon Nov 2 13:55:04 UTC 2020
machine : x86_64
nr_cpus : 4
max_cpu_id : 3
nr_nodes : 1
cores_per_socket : 2
threads_per_core : 2
cpu_mhz : 2999.974
hw_caps : 1f8bfbff:f7fa3203:2c100800:00000121:0000000f:d19f47ab:00000008:00000100
virt_caps : pv shadow
total_memory : 7891
free_memory : 3703
sharing_freed_memory : 0
sharing_used_memory : 0
outstanding_claims : 0
free_cpus : 0
xen_major : 4
xen_minor : 14
xen_extra : .5.24.g87d90d51
xen_version : 4.14.5.24.g87d90d51
xen_caps : xen-3.0-x86_64
xen_scheduler : credit2
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset :
xen_commandline : placeholder dom0_max_vcpus=16 dom0_mem=4096M,max:4096M cpuinfo com1=115200,8n1 console=com1,tty loglvl=all guest_loglvl=all
cc_compiler : gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
cc_compile_by : mockbuild
cc_compile_domain : centos.org
cc_compile_date : Mon Jul 25 08:40:16 UTC 2022
build_id : 00d714f3c5983a534f512ca173dbfdcc7ed5a1b5
xend_config_format : 4
[root@test-xen ~]# cat /proc/cmdline
placeholder nr_cpus=16 root=UUID=44a6a613-4e21-478b-a909-ab653c9d39df ro console=tty0 crashkernel=auto net.ifnames=0 console=ttyS0 console=hvc0 earlyprintk=xen nomodeset
[root@test-xen ~]#

3. With the above config, dom0 only recognized 4 CPUs, and all the other vCPUs are offline:

[root@test-xen ~]# xl vcpu-list
Name ID VCPU CPU State Time(s) Affinity (Hard / Soft)
Domain-0 0 0 3 -b- 14.7 all / all
Domain-0 0 1 3 -b- 11.2 all / all
Domain-0 0 2 0 r-- 12.6 all / all
Domain-0 0 3 2 -b- 10.9 all / all
Domain-0 0 4 - --p 0.0 all / all
Domain-0 0 5 - --p 0.0 all / all
Domain-0 0 6 - --p 0.0 all / all
Domain-0 0 7 - --p 0.0 all / all
Domain-0 0 8 - --p 0.0 all / all
Domain-0 0 9 - --p 0.0 all / all
Domain-0 0 10 - --p 0.0 all / all
Domain-0 0 11 - --p 0.0 all / all
Domain-0 0 12 - --p 0.0 all / all
Domain-0 0 13 - --p 0.0 all / all
Domain-0 0 14 - --p 0.0 all / all
Domain-0 0 15 - --p 0.0 all / all
[root@test-xen ~]#

There are only 4 CPUs under /sys/devices/system/cpu:

[root@test-xen ~]# ls -l /sys/devices/system/cpu
total 0
drwxr-xr-x. 6 root root 0 Nov 13 18:04 cpu0
drwxr-xr-x. 6 root root 0 Nov 13 18:04 cpu1
drwxr-xr-x. 6 root root 0 Nov 13 18:04 cpu2
drwxr-xr-x. 6 root root 0 Nov 13 18:04 cpu3
drwxr-xr-x. 2 root root 0 Nov 13 18:43 hotplug
-r--r--r--. 1 root root 4096 Nov 13 18:43 isolated
-r--r--r--. 1 root root 4096 Nov 13 18:43 kernel_max
-r--r--r--. 1 root root 4096 Nov 13 18:43 modalias
-r--r--r--. 1 root root 4096 Nov 13 18:43 nohz_full
-r--r--r--. 1 root root 4096 Nov 13 18:43 offline
-r--r--r--. 1 root root 4096 Nov 13 18:04 online
-r--r--r--. 1 root root 4096 Nov 13 18:43 possible
drwxr-xr-x. 2 root root 0 Nov 13 18:43 power
-r--r--r--. 1 root root 4096 Nov 13 18:43 present
drwxr-xr-x. 2 root root 0 Nov 13 18:43 smt
-rw-r--r--. 1 root root 4096 Nov 13 18:43 uevent
drwxr-xr-x. 2 root root 0 Nov 13 18:43 vulnerabilities

I have attached dmesg for the kernel and xen. Please let me know if you have any thoughts. Thanks!

Jimmy

On Mon, Nov 13, 2023 at 12:00 AM Jan Beulich <jbeulich@xxxxxxxx> wrote:

On 12.11.2023 21:28, Jimmy Lee wrote:
> Hello Xen experts, I am trying to set dom0_max_vcpus to a number that is
> larger than the number of pcpus. For example, I have a 4-CPU machine but I
> want applications in dom0 to have an illusion that they are running on a
> 64-cpu machine. However, it seems that dom0 will always recognize the
> number of pcpus. I can see the vcpus in the output of "xl vcpu-list" but
> most of them are not online. How can I achieve my goal? Thanks!

Even if I was to assume the Dom0 kernel is Linux, it would still remain
unclear what Linux version you use. Generally with recent Linux kernels
this has been working for me (albeit I didn't try in a while, after
putting together what has become e25a8d959992 ["x86/Xen: streamline (and
fix) PV CPU enumeration"]). If despite having this in your Linux you
still don't see this working, you will want to supply applicable logs.

As an aside - 64 vCPU-s on a 4-pCPU system looks excessive to me.

Jan

> [root@ip-10-0-131-61 cpu3]# xl vcpu-list
>> Name ID VCPU CPU State Time(s)
>> Affinity (Hard / Soft)
>> Domain-0 0 0 0 r-- 15.9 all /
>> all
>> Domain-0 0 1 3 -b- 11.9 all /
>> all
>> Domain-0 0 2 1 -b- 11.2 all /
>> all
>> Domain-0 0 3 2 -b- 12.4 all /
>> all
>> Domain-0 0 4 - --p 0.0 all /
>> all
>> Domain-0 0 5 - --p 0.0 all /
>> all
>> Domain-0 0 6 - --p 0.0 all /
>> all
>> Domain-0 0 7 - --p 0.0 all /
>> all
>> Domain-0 0 8 - --p 0.0 all /
>> all
>> ....
>>
>
> [root@ip-10-0-131-61 cpu3]# lscpu
> Architecture: x86_64
> CPU op-mode(s): 32-bit, 64-bit
> Byte Order: Little Endian
> CPU(s): 4
> On-line CPU(s) list: 0-3
> Thread(s) per core: 4
> Core(s) per socket: 1
> Socket(s): 1
> NUMA node(s): 1
> ...
>
> [root@ip-10-0-131-61 cpu3]# xl info
> ...
> xen_commandline : placeholder dom0_mem=3%,min:2G,max:4G
> com1=115200,8n1 console=com1,tty,vga dom0_max_vcpus=64 spec-ctrl=no
> sched=credit vcpu_migration_delay=1000 allowsuperpage ioapic_ack=new
> ...
>

Re: Faking the number of CPUs for dom0 with dom0_max_vcpus