Ping: [PATCH v12 00/13] Add initial Xen Suspend-to-RAM support on ARM64

[Mykola Kvach <xakep.amatop@xxxxxxxxx>]

Hi all,

Just a gentle ping on this series.

I would appreciate any further review or feedback when you have a chance.

Thanks,
Mykola

On Thu, Aug 27, 2026 at 7:22 PM Mykola Kvach <mykola_kvach@xxxxxxxx> wrote:
>
> This is part 2 of the ARM Xen system suspend/resume patch series, based
> on earlier work by Mirela Simonovic and Mykyta Poturai.
>
> Part 1, covering guest suspend functionality, is already in mainline.
>
> NOTE: Host-wide suspend/resume support is guarded by CONFIG_SYSTEM_SUSPEND,
> which can currently only be selected when UNSUPPORTED is set, and thus the
> host suspend backend is neither enabled by default nor built in supported
> configurations. The separate HAS_HWDOM_SYSTEM_SUSPEND policy bit only changes
> how ARM treats SHUTDOWN_suspend from the hardware domain; it does not enable
> the host-wide suspend backend by itself.
>
> This version is ported to Xen master and includes extensive improvements
> based on reviewer feedback. The patch series restructures code to improve
> robustness and maintainability, and implements initial ARM64 host-wide
> Suspend-to-RAM support driven by control-domain PSCI SYSTEM_SUSPEND
> requests. vPSCI also exposes SYSTEM_SUSPEND as a domain suspend operation
> for all domains; attempt-time host-suspend policy failures are reported as
> PSCI_DENIED rather than hidden through PSCI_FEATURES.
>
> Key updates in this series:
>  - Introduced architecture-specific suspend/resume infrastructure
>  - Integrated GICv2/GICv3 suspend and resume, including memory-backed context
>    save/restore with error handling
>  - Added time and IRQ suspend/resume hooks, ensuring correct timer/interrupt
>    state across suspend cycles
>  - Implemented proper PSCI SYSTEM_SUSPEND invocation and version checks
>  - Added vPSCI SYSTEM_SUSPEND policy for domain suspend and host-wide
>    control-domain sequencing
>  - Improved state management and recovery in error cases during suspend/resume
>  - Added support for IPMMU-VMSA/SMMUv3 context save/restore
>  - Added support for GICv3 eSPI registers context save/restore
>  - Added support for ITS registers context save/restore
> ---
>
> Link to CI: 
> https://gitlab.com/xen-project/people/mykola_kvach/xen/-/pipelines/2796733872
> ---
>
> TODOs:
>  - Enable "xl suspend" support on ARM
>  - Add suspend/resume CI test for ARM (QEMU if feasible)
>  - PCI suspend ?
> ---
>
> Detailed changelogs can be found in each patch.
>
> Changes in v12:
> - Rebase onto the latest master.
> - Updated patch 12; all other patches are unchanged.
>   No functional changes.
>
> Changes in v11:
> - Keep SMMUv3 reset helpers in init text when CONFIG_SYSTEM_SUSPEND is
>   disabled.
> - Update host suspend policy blockers after review: make the runtime gate
>   __ro_after_init, log the SMMUv3 MSI blocker only once, and wrap the Arm
>   IOMMU blocker in CONFIG_SYSTEM_SUSPEND.
>
> Changes in v10:
> - Clarify the vPSCI SYSTEM_SUSPEND policy summary: keep SYSTEM_SUSPEND
>   advertised once implemented and return PSCI_DENIED, rather than
>   PSCI_NOT_SUPPORTED, for attempt-time host-suspend policy failures.
> - Tighten GICv2/GICv3 suspend/resume based on review feedback: avoid
>   reserved interrupt register ranges, check visible active-priority state,
>   restore configuration before enable state, and re-enable the redistributor
>   before restoring CPU/virtual interface state on abort paths.
> - Refine ITS resume so MAPC is replayed only for ITS-backed collections and
>   clarify the collection-ID assumptions.
> - Rework IPMMU and SMMUv3 resume/suspend handling, including root-before-cache
>   IPMMU restore ordering and disabling SMMU interrupt generation before
>   suspend.
> - Save and restore CNTHCTL_EL2 in the arm64 CPU resume context and simplify
>   the resume trampoline/context hand-off.
> - Re-apply boot CPU errata/workaround handling after SYSTEM_SUSPEND and move
>   set_init_ttbr() declaration to asm/mmu/mm.h.
> - Update patch 12 details: shorten SYSTEM_SUSPEND blocker logs, use %pd for
>   control-domain logging, mark serial_suspend_available as __ro_after_init,
>   and mention the xen/suspend.h struct domain forward declaration.
>
> Changes in v9:
> - Split the control-domain SYSTEM_SUSPEND flow so host availability,
>   runtime blockers and domain-readiness checks are handled separately from
>   the host suspend backend.
> - Gate vPSCI SYSTEM_SUSPEND on cached host PSCI support and Xen runtime
>   suspend blockers, and log firmware support during initialization.
> - Fold the arm64 resume trampoline into the CPU context save/restore patch
>   and use asm-offsets-generated RESUME_CTX_* definitions for the assembly
>   save/restore path.
> - Tighten the GICv2/GICv3/ITS/IPMMU/SMMUv3 suspend/resume paths based on
>   review feedback, including state-save/restore fixes and safer failure
>   handling.
> - Reorder the host suspend/resume phases so timer and GIC state are
>   handled with local IRQs disabled and restored before console/IOMMU
>   resume.
>
> Changes in v8:
> - Rebased to latest master and refreshed the series accordingly.
> - Added a new GICv3 patch to tolerate retained redistributor LPI state
>   across CPU_OFF/CPU_ON.
> - GICv2 suspend now disables the CPU interface and distributor before
>   saving state.
> - GICv3 suspend/resume fixes the redistributor base used for LPI state.
> - ITS and SMMUv3 suspend/resume paths were tightened, with safer
>   restore/rollback handling and stricter fatal-error handling.
> - System suspend now checks that all domains are already in
>   SHUTDOWN_suspend before proceeding, and renames the hardware-domain
>   suspend capability/helper for clearer semantics.
> - Fixed alignment/cleanup issues in the low-level suspend/resume code.
>
> Changes in v7:
> - Timer helper renamed/clarified; virtual/hyper/phys handling documented.
> - GICv2 uses one context block; restore saved CTLR; panic on alloc failure.
> - GICv3/eSPI/ITS always suspend/resume; restore LPI/eSPI; rdist timeout.
> - IPMMU suspend context allocated before PCI setup.
> - System suspend: control domain drives host suspend.
> - Dropped v6 IRQ descriptor restore patches; use setup_irq and re-register
>   local IRQs on resume instead.
>
> For earlier changelogs, please refer to the previous cover letters.
>
> Mirela Simonovic (5):
>   xen/arm: Add suspend and resume timer helpers
>   xen/arm: gic-v2: Implement GIC suspend/resume functions
>   xen/arm64: Save/restore CPU context across SYSTEM_SUSPEND
>   xen/arm: Implement PSCI SYSTEM_SUSPEND call (host interface)
>   xen/arm: Add host system suspend backend
>
> Mykola Kvach (7):
>   xen/arm: gic-v3: tolerate retained redistributor LPI state across
>     CPU_OFF
>   xen/arm: gic-v3: Implement GICv3 suspend/resume functions
>   xen/arm: gic-v3: add ITS suspend/resume support
>   xen/arm: tee: keep init_tee_secondary() for hotplug and resume
>   xen/arm: ffa: fix notification SRI across CPU hotplug/suspend
>   xen/arm: smmu-v3: add suspend/resume handlers
>   xen/arm: Add vPSCI SYSTEM_SUSPEND policy
>
> Oleksandr Tyshchenko (1):
>   iommu/ipmmu-vmsa: Implement suspend/resume callbacks
>
>  xen/arch/arm/Kconfig                     |   2 +
>  xen/arch/arm/Makefile                    |   1 +
>  xen/arch/arm/arm64/asm-offsets.c         |  21 +
>  xen/arch/arm/arm64/head.S                | 122 ++++++
>  xen/arch/arm/cpuerrata.c                 |   7 +-
>  xen/arch/arm/gic-v2.c                    | 226 +++++++++++
>  xen/arch/arm/gic-v3-its.c                | 146 ++++++-
>  xen/arch/arm/gic-v3-lpi.c                |  80 +++-
>  xen/arch/arm/gic-v3.c                    | 482 ++++++++++++++++++++++-
>  xen/arch/arm/gic.c                       |  35 ++
>  xen/arch/arm/include/asm/arm64/sysregs.h |   5 +
>  xen/arch/arm/include/asm/cpuerrata.h     |   1 +
>  xen/arch/arm/include/asm/gic.h           |  16 +
>  xen/arch/arm/include/asm/gic_v3_defs.h   |   3 +
>  xen/arch/arm/include/asm/gic_v3_its.h    |  28 ++
>  xen/arch/arm/include/asm/mmu/mm.h        |   2 +
>  xen/arch/arm/include/asm/psci.h          |   4 +
>  xen/arch/arm/include/asm/suspend.h       |  37 ++
>  xen/arch/arm/include/asm/time.h          |   5 +
>  xen/arch/arm/mmu/smpboot.c               |   2 +-
>  xen/arch/arm/psci.c                      |  38 +-
>  xen/arch/arm/suspend.c                   | 210 ++++++++++
>  xen/arch/arm/tee/ffa_notif.c             |  63 ++-
>  xen/arch/arm/tee/tee.c                   |   2 +-
>  xen/arch/arm/time.c                      |  44 ++-
>  xen/arch/arm/vpsci.c                     | 120 +++++-
>  xen/common/Kconfig                       |   3 +
>  xen/common/domain.c                      |   7 +-
>  xen/drivers/char/serial.c                |  12 +
>  xen/drivers/passthrough/arm/iommu.c      |   6 +
>  xen/drivers/passthrough/arm/ipmmu-vmsa.c | 323 ++++++++++++++-
>  xen/drivers/passthrough/arm/smmu-v3.c    | 203 ++++++++--
>  xen/include/xen/list.h                   |  14 +
>  xen/include/xen/serial.h                 |   1 +
>  xen/include/xen/suspend.h                |   2 +
>  35 files changed, 2169 insertions(+), 104 deletions(-)
>  create mode 100644 xen/arch/arm/suspend.c
>
> --
> 2.43.0
>
>