[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v14 04/15] xen/percpu: defer freeing per-CPU areas after resume failures


  • To: Jürgen Groß <jgross@xxxxxxxx>
  • From: Mykola Kvach <xakep.amatop@xxxxxxxxx>
  • Date: Thu, 19 Nov 2026 13:59:36 +0200
  • Arc-authentication-results: i=1; mx.google.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=TTakPD3PLeAFXqmXj5kNB+BekR8BMgNfmWjba8qaf70=; fh=MrI02rc20GP/VrKE8nqr0XJ2mp59Dkr+cJdNDF0uzz4=; b=C8eDsC4p9tSIsPcIOc0wC4fll3dMafa5GiiWl07AV2/HpIp3JrHePMVR7nY/5r7cpH Ali159XbVG+AjSiN7OOkXKOlEXBD+Rj+hMf1P80TcoZS/w6rmE012IFGqUZ2WJo+vyCD M2bObxc+5BjWGMORIZpFLhTyRYomZ/s32Hf8siX4+jM9XNjZxwLw/9f9KKJIBikazLx/ bhmoOqsiB7Mmrtjrg3GDDpfCOAANy2X6W6fQY9nyWE5VfzYgu/8SstbQHFL6DqWGin0f yY6t8/8iR2dVm/mZRmwq4TUkIXGp87HAF9MiThOt4oXYd9UeHLOE2ZGgs2VX9DTalJZd ppOw==; darn=lists.xenproject.org
  • Arc-seal: i=1; a=rsa-sha256; t=1795089591; cv=none; d=google.com; s=arc-20260327; b=f8rNMpdJ9/G8AdFrp13j39/1354Zal4uogZQU/mh3stwdIH1vzk2rhJ42STEKjWvWN IuK4aC7FSosDuC1Q/4x+9OcPkK1W1nYjiBJlDo1uwucURzBwZdrn1FhR/rKM53NCByVZ U4b07vogaooakbhdI87NxKpAZA72/rmtbTGmqBBEmaBTENLjWL0/eQRulY5vzKvGxRXi bM5KSahgI6jv6sOQnf+852NQVdvAHChfZBHkfY+ZwFYtTWpwe3hGdSPlfaOWOO9lqSnA t3vehBrHnpujRmQM8JJwOiagYcIk2OACs4PSm5dCpa62MbdHk+rq7UxH5FzzViTaSDrW 7ycA==
  • Authentication-results: eu.smtp.expurgate.cloud; dkim=pass header.s=20260707 header.d=gmail.com header.i="@gmail.com" header.h="Content-Transfer-Encoding:Content-Type:Cc:To:Subject:Message-ID:Date:From:In-Reply-To:References:MIME-Version"
  • Cc: Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>, Mykola Kvach <Mykola_Kvach@xxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Anthony PERARD <anthony.perard@xxxxxxxxxx>, Michal Orzel <michal.orzel@xxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>, Julien Grall <julien@xxxxxxx>, Roger Pau Monné <roger@xxxxxxxxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>
  • Delivery-date: Thu, 19 Nov 2026 11:59:55 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

Hi Jürgen,

Thank you for the review.

On Thu, Nov 19, 2026 at 1:32 PM Jürgen Groß <jgross@xxxxxxxx> wrote:
>
> On 18.11.26 02:51, Volodymyr Babchuk wrote:
> > Hi,
> >
> > CC'ed Juergen, because I have a question about cpupools.
> >
> > Mykola Kvach <Mykola_Kvach@xxxxxxxx> writes:
> >
> >> When a CPU fails to come back during system resume, cpu_up() sends
> >> CPU_UP_CANCELED. After trying the other CPUs, enable_nonboot_cpus()
> >> sends CPU_RESUME_FAILED for the same CPU.
> >>
> >> On architectures which do not park offline CPUs, both notifications
> >> queue free_percpu_area(). The release metadata, including the rcu_head,
> >> is stored in that CPU's per-CPU area. The first callback may already
> >> have freed the area when the second notification arrives, or the same
> >> rcu_head may be queued twice. The CPU_RESUME_FAILED callbacks also need
> >> the area to clean up timers and CPU-pool state.
>
> I don't see why CPU_RESUME_FAILED would free percpu data twice.
> Even CPU_UP_CANCELED during resuming shouldn't cause a free of percpu
> data.
>
> When CPU_RESUME_FAILED is signaled, system_state should still be
> SYS_STATE_suspend. Or am I overlooking something here?

By the time enable_nonboot_cpus() runs, system_state is already
SYS_STATE_resume. Both enter_state() on x86 and system_suspend() in
Arm patch 15/15 set it before that call.

On x86, park_offline_cpus is true, so the per-CPU notifier skips
free_percpu_area() for both events. On Arm it is false. Without
patch 4, if __cpu_up() fails for a CPU in frozen_cpus, CPU_UP_CANCELED
queues an RCU callback to free that CPU's per-CPU area.

The CPU_RESUME_FAILED handlers still need that area. If the first
callback is still pending, the per-CPU notifier can queue the same
rcu_head again. This patch defers queuing the release until the timer
and cpupool callbacks have finished.

>
> >
> > A bit convoluted. As I see, problem is caused by the fact
> > CPU_UP_CANCELED is a terminal state in a "simple" CPU up sequence, but
> > is a transient state in a system resume sequence. As such, theoretically
> > other per_cpu callbacks might have a similar issue when they don't
> > expect one more "CPU state is bad" callback. I can see that
> > common/timer.c also tries to handle both callbacks as terminal
> > ones. Probably, you need a similar fix there as well.
> >
> > Actually, I am not sure why do we need CPU_RESUME_FAILED in the first
> > place. As I can see, only cpupool handles it in a sensible way. Which
> > begs a question: why cpupool can't call cpupool_cpu_remove_forced() on
> > CPU_UP_CANCELED instead?
> The difference is for the case of parked cpus: CPU_UP_CANCELED might be
> called for parked cpus not coming up, while CPU_RESUME_FAILED will only
> be used for non-parked cpus.

Thanks, that explains why cpupool uses CPU_RESUME_FAILED.

Best regards,
Mykola

>
>
> Juergen



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.