RE: [Xen-devel] [RFC] Physical hot-add cpus and TSC

["Jiang, Yunhong" <yunhong.jiang@xxxxxxxxx>]

>-----Original Message-----
>From: Dan Magenheimer [mailto:dan.magenheimer@xxxxxxxxxx]
>Sent: Thursday, May 27, 2010 11:08 PM
>To: Jiang, Yunhong; Keir Fraser; Xen-Devel (xen-devel@xxxxxxxxxxxxxxxxxxx); Ian
>Pratt
>Subject: RE: [Xen-devel] [RFC] Physical hot-add cpus and TSC
>
>> >From: Keir Fraser [mailto:keir.fraser@xxxxxxxxxxxxx]
>> >
>> >I implemented this as xen-unstable:21468. This represents a strict
>> >improvement on what was in xen-unstable before that (no tsc sync at
>> all,
>> >ever, because I deleted it about a week ago). Open to further
>> improvements,
>> >if we can get consensus.
>>
>> Thanks for the patch. I will get the system that support CPU hotplug
>> tomorrow morning, I can try it then.
>>
>> --jyh
>

Below is test result:
a) With the patch
Before hotadd:
(XEN) TSC marked as reliable, warp = 0 (count=3)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 0 (count=4)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 0 (count=5)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 0 (count=6)
(XEN) No domains have emulated TSC

After add
(XEN) TSC marked as reliable, warp = 1669912421214 (count=15)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 1669912421214 (count=16)
(XEN) No domains have emulated TSC

b) With the patch:
Before adding:
(XEN) TSC marked as reliable, warp = 0 (count=2)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 0 (count=3)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 0 (count=4)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 0 (count=5)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 0 (count=6)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 0 (count=7)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 0 (count=8)
(XEN) No domains have emulated TSC

After add:
(XEN) TSC marked as reliable, warp = 407 (count=12)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 407 (count=13)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 407 (count=14)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 407 (count=15)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 407 (count=16)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 444 (count=17)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 444 (count=18)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 525 (count=19)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 525 (count=20)
(XEN) No domains have emulated TSC
(XEN) TSC marked as reliable, warp = 525 (count=21)
(XEN) No domains have emulated TSC

>Hmmm... I'd be very surprised if this works ever, let alone
>always across all hardware.  By "work", I mean that it will
>result in an "undetectably small difference" (less than a
>cache bounce), as defined and measured by the tsc warp test.
>Why?  Because rdtsc is a long instruction (30-100 cycles)
>and I'll bet writing to the TSC is even longer.  Plus,
>there's a cache bounce overhead added in due to the
>synchronization via the in-memory tsc_count variable.

I think that depends how we define " undetectably". If the time that guest 
migration among physical CPU is much higher than this difference, do we really 
need care about it (of course SMI/NMI is another story)?
Or I missed anything?

--jyh

>
>Our firmware guys say that TSC synchronization can't be
>implemented algorithmically in firmware... it requires
>a simultaneous "reset" signal to all sockets/cores, which
>is obviously not an option here.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel