RE: [Xen-devel] rdtscP and xen (and maybe the app-tsc answer I've been looking for)

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> Why do you need to distinguish between the two emulated rdtscp cases? 
> Special-casing a version of '0' is awkward because it would arise
> naturally from version wraparound (after 2^31 time parameter updates,
> but still).

You're right, I don't need to differentiate between
the two emulated cases.  I was trying to overload
an extra piece of information that I really don't
need to overload.

However, I do need one special case to indicate
emulation vs non-emulation, so wraparound is
still a problem.

Fortunately, wraparound should only occur impossibly
rarely (see below), probably less frequently than
TSC wraparound.

> If the hardware doesn't support rdtscp, how should an app know whether
> or not to use it?  Should it just try running rdtscp being prepared to
> handle a SIGILL?

Yes, that's the plan.  I think this scheme always
works, but only works fast if the hardware supports
rdtscp and constant_tsc.

> If rdtscp is not reliable but Xen has accurate tsc parameter 
> info, then
> the algorithm above will still work efficiently.
> :
> Well, it just needs to increment it whenever Xen knows the tsc has
> changed, as the current pvclock code does.  It could be more 
> frequently
> than restore/migrate if tsc changes on power events.

I've restricted the scheme to constant_tsc as I think
it breaks down due to nasty races if running on a
machine where the pvclock parameters differ across
different pcpus.  I think the races can only be
avoided if Xen sets the TSC_AUX for all of the
pcpus running a pvrdtscp doman while all are idle.

Is there a scheme that avoids the races?

Fortunately, this also has the effect of greatly
reducing the version increase frequency.

> > Also even on machines where TSC is reliable,
> > there is a small chance that consecutive
> > TSC values read will be from different
> > processors and so TSC might appear to go
> > backwards by some small amount.  So apps
> > must still put raw TSC values through
> > a "monotonicity filter".  (Xen already
> > does this for emulated reads of TSC.)
> 
> Why?  I thought "reliable" tscs were supposed to be synced 
> between cores?

The rate is synced but the values may not be.  Since
software (BIOS or Xen) sets tsc on each processor
it is essentially impossible to ensure they are
identical.  The rendezvous algorithm should be able
to set them so that they are "unobservably" different,
but I keep hearing "within 2usec".  (It would be
interesting to measure this across a broad set
of machines.)  So it's probably prudent to recommend
that apps be prepared for the possibility even if
it never happens.

Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel