RE: [Xen-devel] TSC virtualization across different host frequency platform migration

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> that's always the case even w/o migration, since VM can be
> preempted at any time.

Not if it the vcpu is pinned, and that may often
be the case for database apps.

> That's really application specific, depending on the frequency
> of gettimeofday, e.g. in some database transation to stamp
> each record fastly. I had no exact data though. One of my
> colleagues (Xiaowei Yang) once solved one severe performance
> downgrade issue (orders of magnitude, >10%), when guest
> is observed falling back to use vHPET instead of TSC. The
> reason for fallback is not important here, but that actually inspires
> us to pay more attention here.

I'm suggesting that you measure and compare the cycle
cost of a TSC read and a vTSC read (and maybe also
a vHPET read), and also look at the code to see what
the bottleneck is for a vTSC read and if it can be made
faster.  And since your solution doesn't solve PV time,
maybe also measure a vTSC read in a PV guest.

I also agree with Keir that a tsc scaler might be a
good enhancement to request for future VT-x designs.

> -----Original Message-----
> From: Tian, Kevin [mailto:kevin.tian@xxxxxxxxx]
> Sent: Thursday, April 23, 2009 12:15 AM
> To: Dan Magenheimer; Dong, Eddie; xen-devel@xxxxxxxxxxxxxxxxxxx
> Cc: Keir Fraser; Zhang, Xiantao; Yang, Xiaowei
> Subject: RE: [Xen-devel] TSC virtualization across different
> host frequency platform migration
>
>
> >From: Dan Magenheimer [mailto:dan.magenheimer@xxxxxxxxxx]
> >Sent: 2009年4月23日 11:40
> >
> >OK, I think I understand, but I think you are
> >solving a very limited problem ("make sure that
> >a user/program that requests time-of-day gets
> >an answer which is close to accurate") but
> >not solving the broader class of time problems,
> >and you may be making them worse.  If your solution
> >is implemented and the OS or an application reads tsc,
> >the values obtained will be monotonically increasing
> >but will have large gaps, correct?
>
> that's always the case even w/o migration, since VM can be
> preempted at any time.
>
> >
> >If software-emulated tsc reads is really 10% loss
> >in system performance, I agree that this might be
> >the lesser of two evils.  But I don't believe the
> >10%.
>
> That's really application specific, depending on the frequency
> of gettimeofday, e.g. in some database transation to stamp
> each record fastly. I had no exact data though. One of my
> colleagues (Xiaowei Yang) once solved one severe performance
> downgrade issue (orders of magnitude, >10%), when guest
> is observed falling back to use vHPET instead of TSC. The
> reason for fallback is not important here, but that actually inspires
> us to pay more attention here.
>
> Thanks,
> Kevin
>
> >
> >> -----Original Message-----
> >> From: Tian, Kevin [mailto:kevin.tian@xxxxxxxxx]
> >> Sent: Wednesday, April 22, 2009 9:21 PM
> >> To: Dong, Eddie; Dan Magenheimer; xen-devel@xxxxxxxxxxxxxxxxxxx
> >> Cc: Keir Fraser; Zhang, Xiantao
> >> Subject: RE: [Xen-devel] TSC virtualization across different
> >> host frequency platform migration
> >>
> >>
> >> >From: Dong, Eddie
> >> >Sent: 2009年4月23日 9:29
> >> >
> >> >Dan Magenheimer wrote:
> >> >> Hi Eddie/Kevin --
> >> >>
> >> >> I'm sorry to be dense, but I don't understand the
> >> >> details of your solution.  I'm also not sure I
> >> >> understand the problem you are trying to solve.
> >> >> The problem description doesn't describe a problem,
> >> >> just an event.
> >> >>
> >> >> I'm guessing the problem is:  When a guest chooses
> >> >> TSC as its primary clocksource AND a migration later
> >> >> occurs to a host with a different TSC frequency
> >> >> THEN wallclock time (e.g. the "date" command)
> >> >> becomes incorrect.
> >> >
> >> >Mostly yes though don't know if guest wall clock depends on
> >> >TSC heavily.
> >> >
> >> >>
> >> >> I'm also guessing that you are NOT trying to solve
> >> >> the problem:  An application that uses TSC
> >> >> heavily to measure the passage of time AND
> >> >> calibrates TSC on host A AND invisibly
> >> >> migrates to host B with a different TSC
> >> >> frequency THEN will NOT be able to accurately
> >> >> measure the passage of time.  However, it will
> >> >> continue to be monotonically increasing.
> >> >
> >> >Yes, if we don't scale the TSC.
> >> >The proposal tries to solve the problem.
> >> >
> >> >We can use software trap and emulate to scale the TSC so
> >> >that guest TSC after migration is same with that before migration.
> >> >
> >> >But this is not optimial since the overhead may be too high. So we
> >> >propose to use smart scaling, which continuously use TSC_OFFSET,
> >> >but adjust the TSC_OFFSET value time to time (today it is fixed),
> >> >so that an application that uses TSC heavily to measure the
> >> >passage of time
> >> >can get correct time.
> >> >
> >>
> >> So we're really not trying to solve the micro-level accuracy if app
> >> tries to use TSC for that purpose, which always bias from bare
> >> metal as long as running in a VM. Here we're trying to ensure
> >> macro-level accuracy and thus ToD in guest after migration, with
> >> performance optimized if app in VM uses gettimeofday frequently.
> >> In that way, gap between trap vs. notrap would be orders of
> >> magnitude.
> >>
> >> Thanks
> >> Kevin
> >

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel