Re: [PATCH 05/16] xen/percpu: don't initialize percpu on resume

[Jan Beulich <jbeulich@xxxxxxxx>]

On 11.03.2025 21:59, Julien Grall wrote:
> On 05/03/2025 09:11, Mykola Kvach wrote:
>> Invocation of the CPU_UP_PREPARE notification
>> on ARM64 during resume causes a crash:
>>
>> (XEN) [  315.807606] Error bringing CPU1 up: -16
>> (XEN) [  315.811926] Xen BUG at common/cpu.c:258
>> [...]
>> (XEN) [  316.142765] Xen call trace:
>> (XEN) [  316.146048]    [<00000a0000202264>] enable_nonboot_cpus+0x128/0x1ac 
>> (PC)
>> (XEN) [  316.153219]    [<00000a000020225c>] enable_nonboot_cpus+0x120/0x1ac 
>> (LR)
>> (XEN) [  316.160391]    [<00000a0000278180>] 
>> suspend.c#system_suspend+0x4c/0x1a0
>> (XEN) [  316.167476]    [<00000a0000206b70>] 
>> domain.c#continue_hypercall_tasklet_handler+0x54/0xd0
>> (XEN) [  316.176117]    [<00000a0000226538>] 
>> tasklet.c#do_tasklet_work+0xb8/0x100
>> (XEN) [  316.183288]    [<00000a0000226920>] do_tasklet+0x68/0xb0
>> (XEN) [  316.189077]    [<00000a000026e120>] domain.c#idle_loop+0x7c/0x194
>> (XEN) [  316.195644]    [<00000a0000277638>] shutdown.c#halt_this_cpu+0/0x14
>> (XEN) [  316.202383]    [<0000000000000008>] 0000000000000008
>>
>> Freeing per-CPU areas and setting __per_cpu_offset to INVALID_PERCPU_AREA
>> only occur when !park_offline_cpus and system_state is not SYS_STATE_suspend.
>> On ARM64, park_offline_cpus is always false, so setting __per_cpu_offset to
>> INVALID_PERCPU_AREA depends solely on the system state.
>>
>> If the system is suspended, this area is not freed, and during resume, an 
>> error
>> occurs in init_percpu_area, causing a crash because INVALID_PERCPU_AREA is 
>> not
>> set and park_offline_cpus remains 0:
>>
>>      if ( __per_cpu_offset[cpu] != INVALID_PERCPU_AREA )
>>          return park_offline_cpus ? 0 : -EBUSY;
>>
>> It appears that the same crash can occur on x86 if park_offline_cpus is set
>> to 0 during Xen suspend.
> 
> I am rather confused. Looking at the x86 code, it seems 
> park_offline_cpus is cleared for AMD platforms. So are you saying the 
> suspend/resume doesn't work on AMD?

Right now I can't see how it would work there. I've asked Marek for 
clarification
as to their users using S3 only on Intel hardware.

Jan