[xen staging] x86/time: adjust time recording in time_calibration_tsc_rendezvous()

[patchbot@xxxxxxx]

commit a2639824e8458a8f3826e184081a13bfa27ea884
Author:     Jan Beulich <jbeulich@xxxxxxxx>
AuthorDate: Fri Feb 19 17:21:12 2021 +0100
Commit:     Jan Beulich <jbeulich@xxxxxxxx>
CommitDate: Fri Feb 19 17:21:12 2021 +0100

    x86/time: adjust time recording in time_calibration_tsc_rendezvous()
    
    The (stime,tsc) tuple is the basis for extrapolation by get_s_time().
    Therefore the two better get taken as close to one another as possible.
    This means two things: First, reading platform time is too early when
    done on the first iteration. The closest we can get is on the last
    iteration, immediately before telling other CPUs to write their TSCs
    (and then also writing CPU0's). While at the first glance it may seem
    not overly relevant when exactly platform time is read (when assuming
    that only stime is ever relevant anywhere, and hence the association
    with the precise TSC values is of lower interest), both CPU frequency
    changes and the effects of SMT make it unpredictable (between individual
    rendezvous instances) how long the loop iterations will take. This will
    in turn lead to higher an error than neccesary in how close to linear
    stime movement we can get.
    
    Second, re-reading the TSC for local recording is increasing the overall
    error as well, when we already know a more precise value - the one just
    written.
    
    Signed-off-by: Jan Beulich <jbeulich@xxxxxxxx>
    Acked-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
    Release-Acked-by: Ian Jackson <iwj@xxxxxxxxxxxxxx>
---
 xen/arch/x86/time.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/xen/arch/x86/time.c b/xen/arch/x86/time.c
index 23971ed7ac..a894f74b88 100644
--- a/xen/arch/x86/time.c
+++ b/xen/arch/x86/time.c
@@ -1662,11 +1662,12 @@ struct calibration_rendezvous {
 };
 
 static void
-time_calibration_rendezvous_tail(const struct calibration_rendezvous *r)
+time_calibration_rendezvous_tail(const struct calibration_rendezvous *r,
+                                 uint64_t tsc)
 {
     struct cpu_time_stamp *c = &this_cpu(cpu_calibration);
 
-    c->local_tsc    = rdtsc_ordered();
+    c->local_tsc    = tsc;
     c->local_stime  = get_s_time_fixed(c->local_tsc);
     c->master_stime = r->master_stime;
 
@@ -1691,11 +1692,11 @@ static void time_calibration_tsc_rendezvous(void *_r)
             while ( atomic_read(&r->semaphore) != (total_cpus - 1) )
                 cpu_relax();
 
-            if ( r->master_stime == 0 )
-            {
-                r->master_stime = read_platform_stime(NULL);
+            if ( r->master_tsc_stamp == 0 )
                 r->master_tsc_stamp = rdtsc_ordered();
-            }
+            else if ( i == 0 )
+                r->master_stime = read_platform_stime(NULL);
+
             atomic_inc(&r->semaphore);
 
             if ( i == 0 )
@@ -1720,7 +1721,7 @@ static void time_calibration_tsc_rendezvous(void *_r)
         }
     }
 
-    time_calibration_rendezvous_tail(r);
+    time_calibration_rendezvous_tail(r, r->master_tsc_stamp);
 }
 
 /* Ordinary rendezvous function which does not modify TSC values. */
@@ -1745,7 +1746,7 @@ static void time_calibration_std_rendezvous(void *_r)
         smp_rmb(); /* receive signal /then/ read r->master_stime */
     }
 
-    time_calibration_rendezvous_tail(r);
+    time_calibration_rendezvous_tail(r, rdtsc_ordered());
 }
 
 /*
--
generated by git-patchbot for /home/xen/git/xen.git#staging