[PATCH v4][PART 2 00/10] Add initial Xen Suspend-to-RAM support on ARM64

[Mykola Kvach <xakep.amatop@xxxxxxxxx>]

From: Mykola Kvach <mykola_kvach@xxxxxxxx>

This is part 2 of version 4 of the ARM Xen system suspend/resume patch
series, based on earlier work by Mirela Simonovic and Mykyta Poturai.
This version is ported to Xen master (4.21-unstable) and includes
extensive improvements based on reviewer feedback. The patch series
restructures code to improve robustness, maintainability, and implements
system Suspend-to-RAM support on ARM64 hardware domains.

Part 1 of the v4 patch series is available here:
  https://patchew.org/Xen/cover.1748337249.git.mykola._5Fkvach@xxxxxxxx/

This part also depends on the patch that introduces CONFIG_SYSTEM_SUSPEND:
  
https://patchew.org/Xen/cover.1748381788.git.mykola._5Fkvach@xxxxxxxx/eb586049ef5180bb33e9414c4754ee2621a772bc.1748381788.git.mykola._5Fkvach@xxxxxxxx/

Previous versions:
  V1: https://marc.info/?l=xen-devel&m=154202231501850&w=2
  V2: https://marc.info/?l=xen-devel&m=166514782207736&w=2
  V3: https://lists.xen.org/archives/html/xen-devel/2025-03/msg00168.html


At a high-level, this patch series provides:
 - Support for Host system suspend/resume via PSCI SYSTEM_SUSPEND (ARM64)
 - Suspend/resume infrastructure for CPU context, timers, and GICv2
 - Proper error propagation and recovery throughout the suspend/resume flow

Key updates in this series:
 - Introduced architecture-specific suspend/resume infrastructure (new 
`suspend.c`, `suspend.h`, low-level context save/restore in `head.S`)
 - Integrated GICv2 suspend and resume, including memory-backed context 
save/restore with error handling
 - Added time and IRQ suspend/resume hooks, ensuring correct timer/interrupt 
state across suspend cycles
 - Refactored domain shutdown flow to consistently propagate errors from 
arch-specific suspend handlers
 - Implemented proper PSCI SYSTEM_SUSPEND invocation and version checks
 - Improved state management and recovery in error cases during suspend/resume

---
TODOs:
 - Test system suspend with LLC_COLORING enabled and verify functionality
 - Implement IOMMU suspend/resume handlers and integrate them into the
   suspend/resume path
 - Add GICv3 suspend/resume support if required
 - Enable "xl suspend" support on ARM
 - Properly disable Xen timer watchdog from relevant services
 - Add suspend/resume CI test for ARM (QEMU if feasible)
 - Investigate feasibility and need for implementing system suspend on ARM32
---
Changes introduced in V4:
  - Remove the prior tasklet-based workaround in favor of a more
    straightforward and safer solution.
  - Rework the approach by adding explicit system state checks around
    request_irq and release_irq calls; skip these calls during suspend
    and resume states to avoid unsafe memory operations when IRQs are
    disabled.
  - Prevent reinitialization of local IRQ descriptors on system resume.
  - Restore the state of local IRQs during system resume for secondary CPUs.
  - Drop code for saving and restoring VCPU context (see part 1 of the patch
    series for details).
  - Remove IOMMU suspend and resume calls until these features are implemented.
  - Move system suspend logic to arch_domain_shutdown, invoked from
    domain_shutdown.
  - Add console_end_sync to the resume path after system suspend.
  - Drop unnecessary DAIF masking; interrupts are already masked on resume.
  - Remove leftover TLB flush instructions; flushing is handled in enable_mmu.
  - Avoid setting x19 in hyp_resume as it is not required.
  - Replace prepare_secondary_mm with set_init_ttbr, and call it from 
system_suspend.
  - Produce a build-time error for ARM32 when CONFIG_SYSTEM_SUSPEND is enabled.
  - Use register_t instead of uint64_t in the cpu_context structure.
  - Apply minor fixes such as renaming functions, updating comments, and
    modifying commit messages to accurately reflect the changes introduced
    by this patch series.

For earlier changelogs, please refer to the previous cover letters.

Mirela Simonovic (7):
  xen/x86: Move freeze/thaw_domains to common code
  xen/arm: Add suspend and resume timer helpers
  xen/arm: Implement GIC suspend/resume functions (gicv2 only)
  xen/arm: Add support for system suspend triggered by hardware domain
  xen/arm: Implement PSCI SYSTEM_SUSPEND call (host interface)
  xen/arm: Resume memory management on Xen resume
  xen/arm: Save/restore context on suspend/resume

Mykola Kvach (3):
  xen/arm: Prevent crash during disable_nonboot_cpus on suspend
  xen/arm: irq: avoid local IRQ descriptors reinit on system resume
  xen/arm: irq: Restore state of local IRQs during system resume

 xen/arch/arm/Kconfig               |   1 +
 xen/arch/arm/Makefile              |   1 +
 xen/arch/arm/arm64/head.S          | 110 +++++++++++++++++++
 xen/arch/arm/domain.c              |  21 +++-
 xen/arch/arm/gic-v2.c              | 154 ++++++++++++++++++++++++++
 xen/arch/arm/gic.c                 |  35 ++++++
 xen/arch/arm/include/asm/gic.h     |  12 +++
 xen/arch/arm/include/asm/mm.h      |   2 +
 xen/arch/arm/include/asm/psci.h    |   1 +
 xen/arch/arm/include/asm/suspend.h |  44 ++++++++
 xen/arch/arm/include/asm/time.h    |   5 +
 xen/arch/arm/irq.c                 |  48 +++++++++
 xen/arch/arm/mmu/smpboot.c         |   2 +-
 xen/arch/arm/psci.c                |  23 +++-
 xen/arch/arm/suspend.c             | 167 +++++++++++++++++++++++++++++
 xen/arch/arm/tee/ffa_notif.c       |   2 +-
 xen/arch/arm/time.c                |  56 +++++++---
 xen/arch/arm/vpsci.c               |   4 -
 xen/arch/ppc/stubs.c               |   2 +-
 xen/arch/riscv/stubs.c             |   2 +-
 xen/arch/x86/acpi/power.c          |  25 -----
 xen/arch/x86/domain.c              |   4 +-
 xen/common/domain.c                |  34 +++++-
 xen/include/xen/domain.h           |   2 +-
 xen/include/xen/sched.h            |   3 +
 25 files changed, 709 insertions(+), 51 deletions(-)
 create mode 100644 xen/arch/arm/include/asm/suspend.h
 create mode 100644 xen/arch/arm/suspend.c

-- 
2.48.1