[Xen-devel] RE: rdtsc in userspace

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> From: Jeremy Fitzhardinge [mailto:jeremy@xxxxxxxx]
> On 10/23/09 15:51, Dan Magenheimer wrote:
> > In measuring rdtsc usage in the kernel, both Jeremy
> > and I noticed that compiling the kernel causes a
> > large number of userland rdtsc's.  At first I thought
> > that this meant that gcc was using rdtsc, but gcc
> > sources do not show any use of rdtsc.  Next I suspected
> > bash, but it doesn't either.
> 
> I think the dynamic linker may use rdtsc as a piece of randomness for
> randomizing the addresses of libraries and other mappings.

Interesting.  I'll look into that.  I've collected
some other data that might appear somewhat contradictory:
I record the page offset of eip for each emulated rdtsc
and there are about 28 different values collected,
of which about 20 of them increment for each
exec'ed process.  One would think/hope that there
would only be one place where randomness is generated
for this kind of purpose, but I suppose if it is
inlined, it's hard to say.

Anyway, thanks for the tip, I will dig further.

> >   Finally, it appears that
> > the calls are occurring from glibc, and searching for
> > uses of rdtsc, I found that glibc has its
> > own implementation of clock_gettime that uses rdtsc
> > directly!
> >   
> 
> When I run clock_gettime on my system it uses the vsyscall version
> (which is either a syscall or, with my recent patches, all in 
> usermode).

I could be mistaken but I don't think my test guest,
which is EL5u2-32bit (2.6.18-based kernel) has vsyscall.
At least the detection mechanisms that I am aware
of ("sysctl -a | grep vsyscall" and "ls /proc/sys/kernel/vsys*"
don't work.  Is there another way on older kernels?
 
> Using strace on your test program should show whether its really
> bypassing the syscall or not.

You're right!  Strace does show syscalls for
each clock_gettime(), and I was mistaken... these
rdtsc's are in the kernel, not userland.

> > The manpage for clock_gettime ("man 3 clock_gettime")
> > has a lengthy caveat about using clock_gettime on
> > an SMP system BUT provides a mechanism to test to
> > see if your SMP system is safe, clock_getcpuclockid(0).
> 
> My manpages don't have anything like that for clock_gettime().

RHEL5 does.  Though now I am not sure how to use the
section 3 calls.

> That caveat is for CLOCK_PROCESS_CPUTIME_ID which doesn't seem
> reasonable to count on for monotonicity (my manpages
> CLOCK_PROCESS_CPUTIME_ID just refer to as "High resolution per-process
> timer.").   CLOCK_REALTIME or CLOCK_MONOTONIC is a different matter.
> 
> Using what CLOCK_?
> 
> I always see clock_gettime(CLOCK_MONOTONIC or CLOCK_REALTIME, 
> ...) make
> a syscall or vsyscall.  If you're testing any of the 
> thread-local times
> then I think its less interesting.
> 
>     J

I'll dig further next week.

Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel