[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v14 04/15] xen/percpu: defer freeing per-CPU areas after resume failures


  • To: Mykola Kvach <Mykola_Kvach@xxxxxxxx>
  • From: Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>
  • Date: Wed, 18 Nov 2026 01:51:35 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EoT0SB2DS9g0syYnwJ+CH73SuQyEN+E328OgRYzVKB0=; b=a7/CqDHDqncILxybzxHxYVHiuInbYWzyNndbo7pSAstcG7LYxzzyHONyikq9loKq0zP7OI6PqVq4kXvWQrU4kg7EB8l3Pw8zeXY9CDR/TcowWdB7iJh0H1MbGc5tcEZnbEkzH7NymMetBT8IvGYDt2zwAMtU9d8WyVab7s9U8hhvQALIJhn97fAzz2RhGfhPQB7Lf298Q11ImQthiMV63gq/5S9+KgIyaZUAgbM+So/6RHkyjidgc8sOrIgo0j39LP8IkflFZKGRkuPM46RdJDwkbgPeztqdKYkFQ07xN6aQJnqr3QKo1dJLlTxDruThoRWD2QWSNxsOWQ3E/eIUPw==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lazxLLIDSb7aiWUdPhfsdU6yGpm6yEEJArBvlfwRMbmIvEadmiaU1ds0zfdrOcGVCPqjd5FEpazpNoZ/1n0HeuZ89c2FFnT/aeBwMT46g6YKPkwRt/iwZEmSvOxNHDUmXA12Y4ciVtnTCT+OvFuz3JYvp8pJQKWCCbPEYY5czoOBonkrS4b7ml9Q2qzaPbtQq5j+6Rq2AzsvDYgdS/ZXh0e68gGnteviueuvFxFKsCTjH7EgTq77IjBInKvT8QaeUTxzk9ggGJjnp5EB72JHDe2BkUT41MxdTM0WlXbtc13fdmu30TzhvbbXztfA1YB6j2lQj2jGCmEjq1D0sToykg==
  • Authentication-results: eu.smtp.expurgate.cloud; dkim=pass header.s=selector1 header.d=epam.com header.i="@epam.com" header.h="From:Date:Subject:Message-ID:Content-Type:MIME-Version:x-ms-exchange-senderadcheck"
  • Authentication-results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=epam.com;
  • Cc: "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Anthony PERARD <anthony.perard@xxxxxxxxxx>, Michal Orzel <michal.orzel@xxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>, Julien Grall <julien@xxxxxxx>, Roger Pau Monné <roger@xxxxxxxxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Juergen Gross <jgross@xxxxxxxx>
  • Delivery-date: Wed, 18 Nov 2026 01:51:54 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Thread-index: AQHddi+rPrDcM3j2gUuDy+04RxtZEbczEk+A
  • Thread-topic: [PATCH v14 04/15] xen/percpu: defer freeing per-CPU areas after resume failures

Hi,

CC'ed Juergen, because I have a question about cpupools.

Mykola Kvach <Mykola_Kvach@xxxxxxxx> writes:

> When a CPU fails to come back during system resume, cpu_up() sends
> CPU_UP_CANCELED. After trying the other CPUs, enable_nonboot_cpus()
> sends CPU_RESUME_FAILED for the same CPU.
>
> On architectures which do not park offline CPUs, both notifications
> queue free_percpu_area(). The release metadata, including the rcu_head,
> is stored in that CPU's per-CPU area. The first callback may already
> have freed the area when the second notification arrives, or the same
> rcu_head may be queued twice. The CPU_RESUME_FAILED callbacks also need
> the area to clean up timers and CPU-pool state.

A bit convoluted. As I see, problem is caused by the fact
CPU_UP_CANCELED is a terminal state in a "simple" CPU up sequence, but
is a transient state in a system resume sequence. As such, theoretically
other per_cpu callbacks might have a similar issue when they don't
expect one more "CPU state is bad" callback. I can see that
common/timer.c also tries to handle both callbacks as terminal
ones. Probably, you need a similar fix there as well.

Actually, I am not sure why do we need CPU_RESUME_FAILED in the first
place. As I can see, only cpupool handles it in a sensible way. Which
begs a question: why cpupool can't call cpupool_cpu_remove_forced() on
CPU_UP_CANCELED instead?



>
> Skip the per-CPU release for CPU_UP_CANCELED during resume. Let
> CPU_RESUME_FAILED release the area after the other cleanup callbacks.
>
> This prepares the common code for Arm system suspend. x86 parks offline
> CPUs, so the per-CPU notifier does not call free_percpu_area() for either
> CPU_UP_CANCELED or CPU_RESUME_FAILED.
>
> Signed-off-by: Mykola Kvach <mykola_kvach@xxxxxxxx>
> ---
> Changes in V14:
> - Clarify that x86 skips free_percpu_area() for CPU_UP_CANCELED and
>   CPU_RESUME_FAILED.
> - Remove the blank line after fallthrough. No functional change.
>
> Changes in v13:
> - New patch.
> ---
>  xen/common/percpu.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/xen/common/percpu.c b/xen/common/percpu.c
> index f180f37253..757de111db 100644
> --- a/xen/common/percpu.c
> +++ b/xen/common/percpu.c
> @@ -82,6 +82,14 @@ static int cf_check cpu_percpu_callback(
>          break;
>  
>      case CPU_UP_CANCELED:
> +        /*
> +         * On resume, CPU_RESUME_FAILED is sent after all CPUs have been
> +         * tried. Keep the area alive for its cleanup callbacks and let
> +         * that notification release it.
> +         */
> +        if ( system_state == SYS_STATE_resume )
> +            break;
> +        fallthrough;
>      case CPU_DEAD:
>      case CPU_RESUME_FAILED:
>          if ( !park_offline_cpus && system_state != SYS_STATE_suspend )

-- 
WBR, Volodymyr


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.