Re: [Xen-devel] Xen domU Timekeeping (a.k.a TSC/HPET issues)

[Paul Durrant <Paul.Durrant@xxxxxxxxxx>]

IIRC Windows (7) calibrates tsc against RTC at start of day and you can get 
some pretty odd results if a vcpu is descheduled during that calculation. I was 
not aware of the exact scenario you described. You *can* get the equivalent of 
the /usepmtimer boot.ini option on more recent OS by doing:

bcdedit /set useplatformclock true

Enabling viridian timers in Xen may be the way to go though.

  Paul

> -----Original Message-----
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-devel-
> bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Andres Lagar-Cavilla
> Sent: 17 February 2012 16:28
> To: xen-devel@xxxxxxxxxxxxxxxxxxx
> Cc: Ian Campbell; Qrux
> Subject: Re: [Xen-devel] Xen domU Timekeeping (a.k.a TSC/HPET issues)
> 
> > Date: Fri, 17 Feb 2012 12:06:05 +0000
> > From: Ian Campbell <Ian.Campbell@xxxxxxxxxx>
> > To: Qrux <qrux.qed@xxxxxxxxx>
> > Cc: "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
> > Subject: Re: [Xen-devel] Xen domU Timekeeping (a.k.a TSC/HPET issues)
> > Message-ID: <1329480365.3131.50.camel@xxxxxxxxxxxxxxxxxxxxxx>
> > Content-Type: text/plain; charset="UTF-8"
> >
> > I'm afraid I don't know the answer to most of your questions (hence
> > I'm afraid I've trimmed the quotes rather aggressively) but here's
> > some of what I do know.
> 
> I'm gonna add another data point.
> 
> We're seeing the Windows 7 Query Performance Counter get mightily
> confused. People have reported this in Amazon EC2 as well
> https://forums.aws.amazon.com/thread.jspa?threadID=41426
> 
> We've tracked it down to the hpet. Xen schedules an interrupt delivery for
> an hpet tick, but the vcpu is asleep. Could be an admin pause, a sleep on a
> wait queue, paused while qemu does its thing, paused while a mem event is
> processed...
> 
> When the vcpu wakes up it receives the "late" hpet tick. We believe
> Windows 7 QPC also reads the TSC at that point. The TSC kept on ticking
> while the vcpu was paused. Windows does not know what to do about the
> discrepancy and reports a large time leap, usually consistent with a full
> round trip of the 32 bit hpet counter at the Xen-emulated 1/16GHz
> frequency.
> 
> MSDN forums blame "bad hardware" for this. Funny.
> 
> So, we could solve our particular problem if the tsc were not to tick during
> vcpu sleep. And I get an inkling that would help with this post as well. But I
> don't think any of the advertised timer or tsc modes do that.
> 
> Thanks,
> Andres
> 
> 
> 
> I'm not sure this will help with the original post, but there's gotta be
> somebody who
> >
> >> But, practically, is there a safe CPU configuration?
> >
> > I think that part of the problem here is that it is very hard to
> > determine this at the hardware level. There are at least 3 (if not
> > more) CPUID feature bits which say "no really, the TSC is good and
> > safe to use this time, you can rely on that" because they keep
> > inventing new ways to get it wrong.
> >
> > [...]
> >>
> >> Since September, I can't find any further information about this
> >> issue. What is the state of this issue?  The inconsistency I see
> >> right now is this: in the July 2010 TSC discussion, a "Stefano Stabellini"
> >> posted this:
> >>
> >> ====
> >> > /me wonders if timer_mode=1 is the default for xl?
> >> > Or only for xm?
> >>
> >> no, it is not.
> >> Xl defaults to 0 [zero], I am going to change it right now.
> >> ====
> >>
> >> So, it seems like (at least as of July 2010), xl is defaulting to
> >> "timer_mode=1".  That is, assuming that the then-current timer_mode
> >> is the same as present-day tsc_mode.
> >
> > No, I believe they are different things.
> >
> > tsc_mode is to do with the TSC, emulation vs direct exposure etc. Per
> > xen/include/asm-x86/time.h and (in recent xen-unstable) xl.cfg(5)
> >
> > timer_mode is to do with the the way that timer interrupts are
> > injected into the guest. This is described in
> xen/include/public/hvm/params.h.
> > This isn't documented in xl.cfg(5) because I couldn't make head nor
> > tail of the meaning of that header :-(
> >
> >>   In addition, I'm assuming he was changing it from 0 (zero) to 1
> >> (one)--and not some other mode.  But,
> >>
> >>         xen-4.1.2/docs/misc/tscmode.txt
> >
> > Remember that he was referring to timer_mode not tsc_mode...
> >
> >> says:
> >>
> >>         "The default mode (tsc_mode==0) checks TSC-safeness of the
> >> underlying
> >>         hardware on which the virtual machine is launched.  If it is
> >>         TSC-safe, rdtsc will execute at hardware speed; if it is not,
> >> rdtsc
> >>         will be emulated."
> >>
> >> Which implies the default is always 0 (zero).  Which is it?
> >
> > It seems that xl, in xen-unstable, defaults to:
> >     timer_mode = 1
> >     tsc_mode = 0
> > as does 4.1 as far as I can tell via code inspection.
> >
> >> More importantly, is the solution to force tsc_mode=2?
> >
> > IMHO this is safe in most situations unless you are running some sort
> > of workload (e.g. a well known database) which has stringent
> > requirements regarding the TSC for transactional consistency (hence
> > the conservative default).
> >
> >>   If so, under what BIOS/xen-boot-params/dom0-boot-params
> conditions?
> >> And--please excuse my exasperation--but WTH does this have to do with
> >> ext3 versus ext4?  Is ext4 exquisitely sensitive to TSC/HPET
> >> "jumpiness" (if that's even what's happening)?
> >
> > Sorry, I have no idea how/why the filesystem would be related to the
> > TSC.
> >
> > It is possible you are actually seeing two bugs I suppose -- there
> > have been issues relating to ext4 and barriers in some kernel versions
> > (I'm afraid I don't recall the details, the list archives ought to
> > contain something).
> >
> > Ian.
> >
> >
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel