|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [PATCH v14 04/15] xen/percpu: defer freeing per-CPU areas after resume failures
Hi Volodymyr, Thank you for the review. On Wed, Nov 18, 2026 at 3:52 AM Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx> wrote: > > Hi, > > CC'ed Juergen, because I have a question about cpupools. > > Mykola Kvach <Mykola_Kvach@xxxxxxxx> writes: > > > When a CPU fails to come back during system resume, cpu_up() sends > > CPU_UP_CANCELED. After trying the other CPUs, enable_nonboot_cpus() > > sends CPU_RESUME_FAILED for the same CPU. > > > > On architectures which do not park offline CPUs, both notifications > > queue free_percpu_area(). The release metadata, including the rcu_head, > > is stored in that CPU's per-CPU area. The first callback may already > > have freed the area when the second notification arrives, or the same > > rcu_head may be queued twice. The CPU_RESUME_FAILED callbacks also need > > the area to clean up timers and CPU-pool state. > > A bit convoluted. As I see, problem is caused by the fact > CPU_UP_CANCELED is a terminal state in a "simple" CPU up sequence, but > is a transient state in a system resume sequence. Agreed, I'll make this clearer in the description. If no other changes are needed, could we update the commit message when applying the patch, without rerolling the series? > As such, theoretically > other per_cpu callbacks might have a similar issue when they don't > expect one more "CPU state is bad" callback. I can see that > common/timer.c also tries to handle both callbacks as terminal > ones. Probably, you need a similar fix there as well. I checked timer.c. free_percpu_timers() sets ts->heap to dummy_heap after freeing it, so the second call does not free the same allocation again. The per-CPU area itself must remain valid, which this patch ensures. I don't see a need for the same fix there. > > Actually, I am not sure why do we need CPU_RESUME_FAILED in the first > place. As I can see, only cpupool handles it in a sensible way. Which > begs a question: why cpupool can't call cpupool_cpu_remove_forced() on > CPU_UP_CANCELED instead? CPU_RESUME_FAILED runs after all CPUs have been tried, when the available resources are known. This was the reason for adding it in commit 51c79e943fb3. Removing a CPU from its pool can require migrating vCPUs or moving domains to pool0. Also, if CPU_UP_PREPARE fails early, CPU_UP_CANCELED only unwinds part of the notifier chain, so cpupool may not receive it. CPU_RESUME_FAILED runs the full chain. But let's wait for Juergen's input on the cpupool part. Best regards, Mykola > > > > > > > Skip the per-CPU release for CPU_UP_CANCELED during resume. Let > > CPU_RESUME_FAILED release the area after the other cleanup callbacks. > > > > This prepares the common code for Arm system suspend. x86 parks offline > > CPUs, so the per-CPU notifier does not call free_percpu_area() for either > > CPU_UP_CANCELED or CPU_RESUME_FAILED. > > > > Signed-off-by: Mykola Kvach <mykola_kvach@xxxxxxxx> > > --- > > Changes in V14: > > - Clarify that x86 skips free_percpu_area() for CPU_UP_CANCELED and > > CPU_RESUME_FAILED. > > - Remove the blank line after fallthrough. No functional change. > > > > Changes in v13: > > - New patch. > > --- > > xen/common/percpu.c | 8 ++++++++ > > 1 file changed, 8 insertions(+) > > > > diff --git a/xen/common/percpu.c b/xen/common/percpu.c > > index f180f37253..757de111db 100644 > > --- a/xen/common/percpu.c > > +++ b/xen/common/percpu.c > > @@ -82,6 +82,14 @@ static int cf_check cpu_percpu_callback( > > break; > > > > case CPU_UP_CANCELED: > > + /* > > + * On resume, CPU_RESUME_FAILED is sent after all CPUs have been > > + * tried. Keep the area alive for its cleanup callbacks and let > > + * that notification release it. > > + */ > > + if ( system_state == SYS_STATE_resume ) > > + break; > > + fallthrough; > > case CPU_DEAD: > > case CPU_RESUME_FAILED: > > if ( !park_offline_cpus && system_state != SYS_STATE_suspend ) > > -- > WBR, Volodymyr
|
![]() |
Lists.xenproject.org is hosted with RackSpace, monitoring our |