Re: [Xen-devel] Xen 4 TSC problems

[George Dunlap <George.Dunlap@xxxxxxxxxxxxx>]

On Tue, Sep 13, 2011 at 8:16 AM, Philippe Simonet
<philippe.simonet@xxxxxxxxxx> wrote:
> Hi Xen developers
>
> i just would like to inform you that I have exactly the same problem with
> Debian squeeze and xen, with
> 50 seconds time jump on my dom0 and domu. NTP is running on all dom0/domuU,
> clocksource is 'xen'
> everywhere.
>
> some messages :
> syslog :
> Sep 11 13:56:50 dnsit22 kernel: [571603.359863] Clocksource tsc unstable
> (delta = -2999662111513 ns)
>
> xm dmesg :
> ...
> (XEN) Platform timer is 14.318MHz HPET
> ...
> (XEN) Platform timer appears to have unexpectedly wrapped 10 or more times.
> (XEN) TSC marked as reliable, warp = 0 (count=2)
> ...

I haven't been following this conversation, so I don't know if this is
relevant, but I've just discovered this morning that the TSC warp
check in Xen is done at the wrong time (before any secondary cpus are
brought up), and thus always returns warp=0.  I've submitted a patch
to do the check after secondary CPUs are brought up; that should cause
Xen to do periodic synchronization of TSCs when there is drift.

 -George

>
> I had some contact with Olivier Hanesse and it indicates that he doesn't
> have any solution for this problem,
> and all what was proposed in February didn't solved this problem.
>
> all suggestions are welcomed.
>
> Best regards
>
> Philippe
>
>
> config :
> --------------------------------------
> Linux dnsit22.swissptt.ch 2.6.32-5-xen-amd64 #1 SMP Tue Jun 14 12:46:30 UTC
> 2011 x86_64 GNU/Linux
> --------------------------------------
> HP DL385
> --------------------------------------
> vendor_id       : AuthenticAMD
> cpu family      : 16
> model           : 9
> model name      : AMD Opteron(tm) Processor 6174
> stepping        : 1
> cpu MHz         : 3058776.574
> cache size      : 512 KB
> fpu             : yes
> fpu_exception   : yes
> cpuid level     : 5
> wp              : yes
> flags           : fpu de tsc msr pae mce cx8 apic mtrr mca cmov pat clflush
> mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow
> constant_tsc rep_good nonstop_tsc extd_apicid amd_dcm pni cx16 popcnt
> hypervisor lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse
> 3dnowprefetch nodeid_msr
> bogomips        : 4409.03
> TLB size        : 1024 4K pages
> clflush size    : 64
> cache_alignment : 64
> address sizes   : 48 bits physical, 48 bits virtual
> power management: ts ttp tm stc 100mhzsteps hwpstate
> --------------------------------------
>
> --------------------------------------
> PCI :
> 00:00.0 Host bridge: ATI Technologies Inc RD890 Northbridge only dual slot
> (2x16) PCI-e GFX Hydra part (rev 02)
> 00:04.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge (PCI
> express gpp port D)
> 00:06.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge (PCI
> express gpp port F)
> 00:0a.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge (external
> gfx1 port A)
> 00:0b.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge (NB-SB
> link)
> 00:0d.0 PCI bridge: ATI Technologies Inc RD890 PCI to PCI bridge (external
> gfx1 port B)
> 00:11.0 SATA controller: ATI Technologies Inc SB700/SB800 SATA Controller
> [IDE mode]
> 00:12.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0
> Controller
> 00:12.1 USB Controller: ATI Technologies Inc SB700 USB OHCI1 Controller
> 00:12.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller
> 00:13.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0
> Controller
> 00:13.1 USB Controller: ATI Technologies Inc SB700 USB OHCI1 Controller
> 00:13.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller
> 00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3d)
> 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
> 00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
> 00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
> 00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> HyperTransport Configuration
> 00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> Address Map
> 00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor DRAM
> Controller
> 00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> Miscellaneous Control
> 00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor Link
> Control
> 00:19.0 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> HyperTransport Configuration
> 00:19.1 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> Address Map
> 00:19.2 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor DRAM
> Controller
> 00:19.3 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> Miscellaneous Control
> 00:19.4 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor Link
> Control
> 00:1a.0 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> HyperTransport Configuration
> 00:1a.1 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> Address Map
> 00:1a.2 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor DRAM
> Controller
> 00:1a.3 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> Miscellaneous Control
> 00:1a.4 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor Link
> Control
> 00:1b.0 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> HyperTransport Configuration
> 00:1b.1 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> Address Map
> 00:1b.2 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor DRAM
> Controller
> 00:1b.3 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor
> Miscellaneous Control
> 00:1b.4 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor Link
> Control
> 01:03.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)
> 02:00.0 System peripheral: Hewlett-Packard Company iLO3 Slave
> instrumentation & System support (rev 04)
> 02:00.2 System peripheral: Hewlett-Packard Company iLO3 Management Processor
> Support and Messaging (rev 04)
> 02:00.4 USB Controller: Hewlett-Packard Company Proliant iLO2/iLO3 virtual
> USB controller (rev 01)
> 03:00.0 RAID bus controller: Hewlett-Packard Company Smart Array G6
> controllers (rev 01)
> 04:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709
> Gigabit Ethernet (rev 20)
> 04:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709
> Gigabit Ethernet (rev 20)
> 05:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709
> Gigabit Ethernet (rev 20)
> 05:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709
> Gigabit Ethernet (rev 20)
> 09:00.0 PCI bridge: PLX Technology, Inc. PEX 8616 16-lane, 4-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev bb)
> 0a:04.0 PCI bridge: PLX Technology, Inc. PEX 8616 16-lane, 4-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev bb)
> 0a:05.0 PCI bridge: PLX Technology, Inc. PEX 8616 16-lane, 4-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev bb)
> 0a:06.0 PCI bridge: PLX Technology, Inc. PEX 8616 16-lane, 4-Port PCI
> Express Gen 2 (5.0 GT/s) Switch (rev bb)
> 0c:00.0 Ethernet controller: Intel Corporation 82580 Gigabit Network
> Connection (rev 01)
> 0c:00.1 Ethernet controller: Intel Corporation 82580 Gigabit Network
> Connection (rev 01)
> 0c:00.2 Ethernet controller: Intel Corporation 82580 Gigabit Network
> Connection (rev 01)
> 0c:00.3 Ethernet controller: Intel Corporation 82580 Gigabit Network
> Connection (rev 01)
> 0f:00.0 Ethernet controller: Intel Corporation 82580 Gigabit Network
> Connection (rev 01)
> 0f:00.1 Ethernet controller: Intel Corporation 82580 Gigabit Network
> Connection (rev 01)
> 0f:00.2 Ethernet controller: Intel Corporation 82580 Gigabit Network
> Connection (rev 01)
> 0f:00.3 Ethernet controller: Intel Corporation 82580 Gigabit Network
> Connection (rev 01)
> --------------------------------------
>
>
>
>
> On 8:59 PM, Olivier Hanesse wrote:
>>
>> Hello
>>
>> I've got an issue about time keeping with Xen 4.0 (Debian squeeze
>> release).
>>
>> My problem is here (hopefully I amn't the only one, so there might be a
>> bug
>> somewhere) : http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=599161#50
>> After some times,  I got this error : Clocksource tsc unstable (delta =
>> -2999660334211 ns). It has happened on several servers.
>>
>> Looking at the output of "xm debug-key s;"
>>
>> (XEN) TSC has constant rate, deep Cstates possible, so not reliable,
>> warp=2850 (count=3)
>>
>> I am using a "Intel(R) Xeon(R) CPU L5420  @ 2.50GHz", which has the
>> "constant_tsc", but not the "nonstop_tsc" one.
>> On other systems with a newer cpu with "nonstop_tsc", I don't have this
>> issue (systems are running the same distros with same config).
>>
>> I tried to boot with "max_cstate=0", but nothing changed, my TSC isn't
>> reliable and after some times, I will got the "50min" issue again.
>>
>> I don't understand how a system can do a jump of "50min" in the future.
>> Why
>> 50min ? it is not 40min, not 1 hour, it is always 50min.
>> I don't know how to make my TSC "reliable" (I already disable everything
>> about Powerstate in BIOS Settings).
>>
>> Any ideas ?
>>
>> Regards
>>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel