Re: [Xen-devel] Ever increasing time offset for HVM domain / Huge amounts of drift

[Phil Evans <Phil.Evans@xxxxxxxx>]

I had tried messing with all but tsc_mode.  I have tried all options for 
tsc_mode as well now but there is no change.  The key thing here is that the 
drift is increasing at a rate of 1 second per second of uptime amounting to 
huge amounts.  It doesn't seem to be a clock skew issue, it seems to be 
something is simply not counting at all.

Phil.
________________________________________
From: Pasi Kärkkäinen [pasik@xxxxxx]
Sent: 14 January 2013 16:30
To: Phil Evans
Cc: xen-devel@xxxxxxxxxxxxx
Subject: Re: [Xen-devel] Ever increasing time offset for HVM domain / Huge 
amounts of drift

On Mon, Jan 14, 2013 at 04:17:39PM +0000, Phil Evans wrote:
> Hi,
>
> Sorry I should have included that in the first place:
>

Ok. Did you try experimenting with these options?:

timer_mode=X
hpet=0|1
tsc_mode=X


-- Pasi


> import os,re
> arch = os.uname()[4]
> kernel = '/usr/lib/xen-default/boot/hvmloader'
> builder = 'hvm'
> name = 'vm_141'
> memory = '2048'
> disk = 
> ['phy:/dev/storage_node_2/disk_806,xvda,w','file:/control/isos/empty.iso,xvdd:cdrom,r']
> vif = ['mac=00:16:3e:3f:2f:1a, bridge=vlan_369, vifname=vm_141.0, 
> ip=89.238.190.22 2a02:40:501:3::2 2a02:40:501:3::5 
> 89.238.190.88','mac=00:16:3e:67:66:71, bridge=vlan_4000, vifname=vm_141.1, 
> ip=0.0.0.0','mac=00:16:3e:01:7e:e8, bridge=vlan_369, vifname=vm_141.2']
> device_model = '/usr/lib/xen-default/bin/qemu-dm'
> boot = 'cd'
> vnc = 1
> vncpasswd = 'YpD5aVZ8'
> usbdevice = 'tablet'
> acpi = 0
> vcpus = 4
> viridian = 1
>
> Thanks,
> Phil.
> ________________________________________
> From: Pasi Kärkkäinen [pasik@xxxxxx]
> Sent: 14 January 2013 15:14
> To: Phil Evans
> Cc: xen-devel@xxxxxxxxxxxxx
> Subject: Re: [Xen-devel] Ever increasing time offset for HVM domain / Huge 
> amounts of drift
>
> On Mon, Jan 14, 2013 at 01:37:01PM +0000, Phil Evans wrote:
> > Hi,
> >
> > I am currently running Xen 4.2.1 (this has also happened in 4.2.0 as well). 
> >  We have been having a major problem with sometimes huge amounts of clock 
> > drift in Windows VMs.  Sometimes the clock on a VM could suddenly jump by 
> > over a week (usually forwards, however time has been known to go backwards 
> > as well).
> >
> > Now I don?t profess to know the internals of Xen, however through my 
> > investigation I believe I have a degree of knowledge of what could be 
> > causing the problem.
> >
> > The steps to reproduce this (for me at least), is to simply do a manual NTP 
> > sync on a Windows VM.  Upon monitoring the qemu-dm log file for the VM, I 
> > see similar to the following:
> >
> > Time offset set 489, added offset 480
> > Time offset set 436, added offset -53
> > Time offset set 496, added offset 60
> > Time offset set 494, added offset -2
> > Time offset set 554, added offset 60
> > Time offset set 565, added offset 11
> > Time offset set 606, added offset 41
> > Time offset set -1974, added offset -2580
> > Time offset set 1626, added offset 3600
> > Time offset set 1579, added offset -47
> > Time offset set 1639, added offset 60
> >
> > It seems to add the same number of seconds to the offset as has passed 
> > since the last sync.  The offset just keeps on increasing, eventually 
> > resulting in huge numbers equating to days.  Occasionally the offset may 
> > jump a bit and go down but the general trend is up.  Although this does not 
> > affect the VM immediately, at some point I am guessing it syncs itself with 
> > the CMOS clock (which is now a large number of seconds offset from the 
> > actual time), resulting in a huge jump in time.  A reboot is a guaranteed 
> > way to get the new, incorrect time.
> >
> > Although I do not understand all of the underlying code, I presume the 
> > correct way this should work is it should be comparing the CMOS time that?s 
> > just been set with the hardware clock on the physical machine, resulting in 
> > an offset between the two.  This would result in a generally stable number 
> > (ideally 0).  Obviously it is incorrect behaviour for the number to keep 
> > going up.  To my mind it looks like it may be somehow getting an inaccurate 
> > time from the system (in many cases a fixed time rather than an up-to-date 
> > current time).
> >
> > Does anyone have any light they may be able to shed on this?  Is it 
> > possible it could be struggling to get an accurate time from the hardware?  
> > I have checked on several occasions and both the system time and the BIOS 
> > clock are spot on.
> >
>
> Please paste the cfgfile of your HVM Windows.
>
> -- Pasi
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel