[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Ever increasing time offset for HVM domain / Huge amounts of drift

  • To: "xen-devel@xxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxx>
  • From: Phil Evans <Phil.Evans@xxxxxxxx>
  • Date: Thu, 10 Jan 2013 18:27:38 +0000
  • Accept-language: en-GB, en-US
  • Delivery-date: Thu, 10 Jan 2013 18:28:52 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xen.org>
  • Thread-index: Ac3vYDcU9aIes6B6QBalEFtVIOTgmg==
  • Thread-topic: Ever increasing time offset for HVM domain / Huge amounts of drift



I am currently running Xen 4.2.1 (this has also happened in 4.2.0 as well).  We have been having a major problem with sometimes huge amounts of clock drift in Windows VMs.  Sometimes the clock on a VM could suddenly jump by over a week (usually forwards, however time has been known to go backwards as well).


Now I don’t profess to know the internals of Xen, however through my investigation I believe I have a degree of knowledge of what could be causing the problem.


The steps to reproduce this (for me at least), is to simply do a manual NTP sync on a Windows VM.  Upon monitoring the qemu-dm log file for the VM, I see similar to the following:


Time offset set 489, added offset 480

Time offset set 436, added offset -53

Time offset set 496, added offset 60

Time offset set 494, added offset -2

Time offset set 554, added offset 60

Time offset set 565, added offset 11

Time offset set 606, added offset 41

Time offset set -1974, added offset -2580

Time offset set 1626, added offset 3600

Time offset set 1579, added offset -47

Time offset set 1639, added offset 60


It seems to add the same number of seconds to the offset as has passed since the last sync.  The offset just keeps on increasing, eventually resulting in huge numbers equating to days.  Occasionally the offset may jump a bit and go down but the general trend is up.  Although this does not affect the VM immediately, at some point I am guessing it syncs itself with the CMOS clock (which is now a large number of seconds offset from the actual time), resulting in a huge jump in time.  A reboot is a guaranteed way to get the new, incorrect time.


Although I do not understand all of the underlying code, I presume the correct way this should work is it should be comparing the CMOS time that’s just been set with the hardware clock on the physical machine, resulting in an offset between the two.  This would result in a generally stable number (ideally 0).  Obviously it is incorrect behaviour for the number to keep going up.  To my mind it looks like it may be somehow getting an inaccurate time from the system (in many cases a fixed time rather than an up-to-date current time).


Does anyone have any light they may be able to shed on this?  Is it possible it could be struggling to get an accurate time from the hardware?  I have checked on several occasions and both the system time and the BIOS clock are spot on.




Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.