[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Xen-devel] [LINUX PATCH 2/2] xen/pvhvm: Support more than 32 VCPUs when migrating.
From: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx> When Xen migrates an HVM guest, by default its shared_info can only hold up to 32 CPUs. As such the hypercall VCPUOP_register_vcpu_info was introduced which allowed us to setup per-page areas for VCPUs. This means we can boot PVHVM guest with more than 32 VCPUs. During migration the per-cpu structure is allocated freshly by the hypervisor (vcpu_info_mfn is set to INVALID_MFN) so that the newly migrated guest can make an VCPUOP_register_vcpu_info hypercall. Unfortunatly we end up triggering this condition in Xen: /* Run this command on yourself or on other offline VCPUS. */ if ( (v != current) && !test_bit(_VPF_down, &v->pause_flags) ) which means we are unable to setup the per-cpu VCPU structures for running vCPUS. The Linux PV code paths make this work by iterating over every vCPU with: 1) is target CPU up (VCPUOP_is_up hypercall?) 2) if yes, then VCPUOP_down to pause it. 3) VCPUOP_register_vcpu_info 4) if it was down, then VCPUOP_up to bring it back up But since VCPUOP_down, VCPUOP_is_up, and VCPUOP_up are not allowed on HVM guests we can't do this. However with the Xen commit "hvm: Support more than 32 VCPUS when migrating." we can do this. As such first check if VCPUOP_is_up is actually possible before trying this dance. As most of this dance code is done already in 'xen_setup_vcpu' lets make it callable on both PV and HVM. This means moving one of the checks out to 'xen_setup_runstate_info'. Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx> --- arch/x86/xen/enlighten.c | 21 ++++++++++++++++----- arch/x86/xen/suspend.c | 6 +----- arch/x86/xen/time.c | 3 +++ 3 files changed, 20 insertions(+), 10 deletions(-) diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c index 201d09a..af8be96 100644 --- a/arch/x86/xen/enlighten.c +++ b/arch/x86/xen/enlighten.c @@ -236,12 +236,23 @@ static void xen_vcpu_setup(int cpu) void xen_vcpu_restore(void) { int cpu; + bool vcpuops = true; + const struct cpumask *mask; - for_each_possible_cpu(cpu) { + mask = xen_pv_domain() ? cpu_possible_mask : cpu_online_mask; + + /* Only Xen 4.5 and higher supports this. */ + if (HYPERVISOR_vcpu_op(VCPUOP_is_up, smp_processor_id(), NULL) == -ENOSYS) + vcpuops = false; + + for_each_cpu(cpu, mask) { bool other_cpu = (cpu != smp_processor_id()); - bool is_up = HYPERVISOR_vcpu_op(VCPUOP_is_up, cpu, NULL); + bool is_up = false; + + if (vcpuops) + is_up = HYPERVISOR_vcpu_op(VCPUOP_is_up, cpu, NULL); - if (other_cpu && is_up && + if (vcpuops && other_cpu && is_up && HYPERVISOR_vcpu_op(VCPUOP_down, cpu, NULL)) BUG(); @@ -250,7 +261,7 @@ void xen_vcpu_restore(void) if (have_vcpu_info_placement) xen_vcpu_setup(cpu); - if (other_cpu && is_up && + if (vcpuops && other_cpu && is_up && HYPERVISOR_vcpu_op(VCPUOP_up, cpu, NULL)) BUG(); } @@ -1751,7 +1762,7 @@ void __ref xen_hvm_init_shared_info(void) for_each_online_cpu(cpu) { /* Leave it to be NULL. */ if (cpu >= MAX_VIRT_CPUS) - continue; + per_cpu(xen_vcpu, cpu) = NULL; /* Triggers xen_vcpu_setup.*/ per_cpu(xen_vcpu, cpu) = &HYPERVISOR_shared_info->vcpu_info[cpu]; } } diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c index 45329c8..6fb3298 100644 --- a/arch/x86/xen/suspend.c +++ b/arch/x86/xen/suspend.c @@ -33,11 +33,7 @@ void xen_arch_hvm_post_suspend(int suspend_cancelled) xen_hvm_init_shared_info(); xen_callback_vector(); xen_unplug_emulated_devices(); - if (xen_feature(XENFEAT_hvm_safe_pvclock)) { - for_each_online_cpu(cpu) { - xen_setup_runstate_info(cpu); - } - } + xen_vcpu_restore(); #endif } diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 7b78f88..d4feb2e 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -105,6 +105,9 @@ void xen_setup_runstate_info(int cpu) { struct vcpu_register_runstate_memory_area area; + if (xen_hvm_domain() && !(xen_feature(XENFEAT_hvm_safe_pvclock))) + return; + area.addr.v = &per_cpu(xen_runstate, cpu); if (HYPERVISOR_vcpu_op(VCPUOP_register_runstate_memory_area, -- 1.7.7.6 _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
Lists.xenproject.org is hosted with RackSpace, monitoring our |