RE: [Xen-devel] RE: Guest TSC and Xen (Intel and AMD feedback please)

["Dan Magenheimer" <dan.magenheimer@xxxxxxxxxx>]

Hi Kevin --

> a) user may mark TSC unstable to migrate among boxes with
> mismatching TSC bits (bus crystal, cpu freq impact, etc.)
>
> b) user may always use TSC as clocksource and then trap RDTSC
> when migrating to a box with mismatching TSC bits
>
> c) user may always use TSC as clocksource when migrating to a
> box with same TSC bits, by adjusting TSC offset
>
> d) migration may be prevented since no reliable methods to ensure
> a)'s effect. Such prevention then falls into generic CPUID comparison
> involved in migration

I think this is an excellent summary of the choices.

> >> I don't know why you want to single out TSC here.
> >
> >I'm singleing it out because it is a per-cpu clock rather
> >than a platform timer... a platform timer can be (and indeed
> >is) offset'ed on migration and that is sufficient if it is
> >selected as the clocksource.
>
> The problem is not per-cpu vs platform, IMO. Instead, it's the
> problem that currently guest TSC is conveyed by host TSC plus
> an offset approach, without read trap. If you also virtualize a
> platform clocksource by a real one, like dedicating a HPET ch,
> same concern also raises.

Yes, you are right.

> >> That is what Linux is testing for anyways. If it decides it is
> >> ok it is fine.
> >
> >Not sure... if Linux thinks it is running on a uniprocessor,
> >but Xen reschedules this uniprocessor Linux guest on a different
> >processor on the same physical SMP system, does Xen adjust the
> >potential TSC difference?  I could be wrong, but I think not.
>
> Xen can do and should be, since SMP system is driven by same
> crystal and thus host TSC is synced. But I guess by far Xen hasn't
> do it, since the TSC drift (dozen of cycles) is smaller than
> the overhead
> to migrate a vcpu. Thus guest won't observe a backward value in
> theory.

I'm not sure this is always the case (though the patch I posted
earlier today may indicate there is something else going on that
has led to my assumption that TSC was skewing worse than dozens
of cycles).

> >> The reason why it is an advantage to try to make TSC btw
> >> is that it is *much* faster than any other timer and there
> >> are definitely workloads that are very timer intensive.
>
> Curiously, how much downgrade using a platform clock source may be,
> for a time-intensive workload?

A good question.  We have a workload that spends >10% of its time
doing gettimeoffset_tsc()... not sure if that is realistic but
it would be interesting to measure that if it used an hpet instead
or if rdtsc was fully emulated.

> >Yes, understood, but if a timer-intensive application makes
> >the assumption that TSC is synchronized and thus will never
> >go backwards, but TSC is not synchronized and it DOES (apparently)
> >go backwards due to Xen scheduler or migration, a slower timer
> >might have been preferred.
>
> Shouldn't this be a software bug instead?

If the application is smart enough to check the TSC bits when
it launches, but stability changes later due to migration/scheduling,
I'm not sure htis is an application bug.  Or did I misunderstand
your question?

Thanks,
Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel