[PATCH v4] 00/30] Cleaning up the KVM clock mess

[David Woodhouse <dwmw2@xxxxxxxxxxxxx>]

This is v4 of the series to clean up the KVM clock, addressing review
feedback from Sean Christopherson and Paul Durrant on v3, rebased to
the current kernel, and incorporating related work from Dongli Zhang.

The KVM clock has historically suffered from three problems:

 1. Imprecision: get_kvmclock_ns() computed the clock from the *host*
    TSC without applying guest TSC scaling, causing systemic drift from
    the values the guest computes from its own TSC.

 2. Unnecessary discontinuities: gratuitous KVM_REQ_MASTERCLOCK_UPDATE
    requests caused the master clock reference point to be re-snapshotted,
    yanking the guest's clock due to arithmetic precision differences.

 3. No precise migration API: the existing KVM_[GS]ET_CLOCK only allows
    setting the clock at a given UTC reference time, which is necessarily
    imprecise. There was no way to preserve the exact arithmetic
    relationship between guest TSC and KVM clock across live migration.

This series addresses all three, and adds new APIs for precise clock
migration and TSC frequency reporting.

Changes since v3:
 - Rebased to v7.1-rc2
 - Split patch 09 (__get_kvmclock fix) into 6 incremental patches per
   Sean's review
 - Split patch 10 (TSC upscaling) into 2 patches per Sean's review
 - Split patch 15 (offset TSCs) into frequency-match vs offset-match
 - Addressed Sean's review: hw_tsc_hz overflow (u64), KVM_VCPU_TSC_SCALE
   gated on has_tsc_control, pvclock_gtod_notifier unregister path,
   kvm_get_time_scale() readability, and many more
 - Incorporated Dongli Zhang's masterclock drift mitigation, reworked as
   a proper deduplication of redundant updates via request clearing under
   the tsc_write_lock
 - Added KVM_VCPU_TSC_EFFECTIVE_FREQ attribute for userspace to populate
   CPUID timing leaves without KVM modifying guest CPUID at runtime
 - Removed runtime Xen TSC CPUID modification (was updating wrong leaf)
 - Added guest-side patches to use CPUID 0x40000010 for TSC frequency
   under both KVM and Xen
 - Selftest covers clock correction at multiple TSC frequencies,
   PVCLOCK_TSC_STABLE_BIT behaviour, and multi-vCPU offset scenarios
 - Fixed RCU splat in KVM_GET_CLOCK_GUEST (needs srcu_read_lock)

The series can be broadly grouped as:

Patches 1-5: Core clock fixes and new KVM_[GS]ET_CLOCK_GUEST API
Patches 6-8: TSC scaling prerequisites
Patches 9-14: Fix get_kvmclock() precision (split per review)
Patches 15-16: Fix kvm_guest_time_update() for TSC upscaling
Patches 17-20: Code cleanup and simplification
Patches 21-22: Allow master clock with offset TSCs
Patches 23-24: Eliminate gratuitous clock updates
Patch 25: Xen runstate negative time fix
Patch 26: Deduplicate redundant masterclock updates
Patches 27-28: TSC frequency reporting for CPUID
Patches 29-30: Guest-side CPUID frequency consumption

David Woodhouse (27):
      KVM: x86/xen: Do not corrupt KVM clock in kvm_xen_shared_info_init()
      KVM: x86: Improve accuracy of KVM clock when TSC scaling is in force
      KVM: x86: Explicitly disable TSC scaling without CONSTANT_TSC
      KVM: x86: Add KVM_VCPU_TSC_SCALE and fix the documentation on TSC 
migration
      KVM: x86: Avoid NTP frequency skew for KVM clock on 32-bit host
      KVM: x86: WARN if kvm_get_walltime_and_clockread() fails unexpectedly
      KVM: x86: Fold __get_kvmclock() into get_kvmclock()
      KVM: x86: Add WARN and restructure get_kvmclock()
      KVM: x86: Use get_kvmclock_base_ns() as fallback in get_kvmclock()
      KVM: x86: Fix KVM clock precision in get_kvmclock() with TSC scaling
      KVM: x86: Use get_kvmclock() in kvm_get_wall_clock_epoch()
      KVM: x86: Fix compute_guest_tsc() to handle negative time deltas
      KVM: x86: Restructure kvm_guest_time_update() for TSC upscaling
      KVM: x86: Simplify and comment kvm_get_time_scale()
      KVM: x86: Remove implicit rdtsc() from kvm_compute_l1_tsc_offset()
      KVM: x86: Improve synchronization in kvm_synchronize_tsc()
      KVM: x86: Kill last_tsc_{nsec,write,offset} fields
      KVM: x86: Replace nr_vcpus_matched_tsc count with all_vcpus_matched_tsc 
bool
      KVM: x86: Allow KVM master clock mode when TSCs are offset from each other
      KVM: x86: Factor out kvm_use_master_clock()
      KVM: x86: Avoid gratuitous global clock updates
      KVM: x86/xen: Prevent runstate times from becoming negative
      KVM: x86: Avoid redundant masterclock updates from multiple vCPUs
      KVM: x86: Add KVM_VCPU_TSC_EFFECTIVE_FREQ attribute
      KVM: x86: Remove runtime Xen TSC frequency CPUID update
      x86/kvm: Obtain TSC frequency from CPUID if present
      x86/xen: Obtain TSC frequency from CPUID if present

Jack Allister (3):
      UAPI: x86: Move pvclock-abi to UAPI for x86 platforms
      KVM: x86: Add KVM_[GS]ET_CLOCK_GUEST for accurate KVM clock migration
      KVM: selftests: Add KVM/PV clock selftest to prove timer correction

 Documentation/virt/kvm/api.rst                 |  37 ++
 Documentation/virt/kvm/devices/vcpu.rst        |  69 ++-
 MAINTAINERS                                    |   4 +-
 arch/x86/include/asm/kvm_host.h                |  13 +-
 arch/x86/include/asm/kvm_para.h                |   1 +
 arch/x86/include/uapi/asm/kvm.h                |  12 +
 arch/x86/include/uapi/asm/kvm_para.h           |  11 +
 arch/x86/include/{ => uapi}/asm/pvclock-abi.h  |  27 +-
 arch/x86/kernel/kvm.c                          |  10 +
 arch/x86/kernel/kvmclock.c                     |   7 +-
 arch/x86/kvm/cpuid.c                           |  16 -
 arch/x86/kvm/svm/svm.c                         |   3 +-
 arch/x86/kvm/vmx/vmx.c                         |   2 +-
 arch/x86/kvm/x86.c                             | 735 ++++++++++++++++++-------
 arch/x86/kvm/xen.c                             |  21 +-
 arch/x86/kvm/xen.h                             |  13 -
 arch/x86/xen/time.c                            |  12 +
 include/uapi/linux/kvm.h                       |   3 +
 tools/testing/selftests/kvm/Makefile.kvm       |   1 +
 tools/testing/selftests/kvm/x86/pvclock_test.c | 415 ++++++++++++++
 20 files changed, 1157 insertions(+), 255 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86/pvclock_test.c
 rename arch/x86/include/{asm => uapi/asm}/pvclock-abi.h (82%)