[Xen-devel] ftrace_enabled set to 1 on bootup, slow downs with CONFIG_FUNCTION_TRACER in virt environments?

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

Hey,

I was running some benchmarks (netserver/netperf) where the init script just 
launched
the netserver and nothing else and was concerned to see the performance not up 
to par.
This was an HVM guest running with PV drivers.

If I compile the kernel without CONFIG_FUNCTION_TRACER it is much better - but 
it was
my understanding that the tracing code does not impact the machine unless it is 
enabled.
And when I inserted a bunch of print_dump_bytes I do see instructions such as
e8 6a 90 60 e1 get replaced with 66 66 66 90 so I see the the instructions 
getting
patched over.

To get a better feel for this I tried this on baremetal, and (this is going
to sound a bit round-about way, but please bear with me), I was working on 
making
the pte_flags be paravirt (so it is a function instead of being a macro) and 
noticed
that on on an AMD A8-3850, with a CONFIG_PARAVIRT and CONFIG_FUNCTION_TRACER and
running kernelbench it would run slower than without CONFIG_FUNCTION_TRACER.

I am not really sure what the problem is, but based on those experiments
four things come to my mind:
 - Lots of nops and we choke the CPU instruction decoder with 20-30 bytes
   of 'nop', so the CPU is stalling waiting for some real instructions.
 - The compiler has choosen to compile most of the paravirt instructions as
   functions making the call to mcount (which gets patched over), but the
   end result is that we have an extra 'call' in the chain.
 - Somehow the low-level para-virt (like the assembler ones) calls don't get
   patched over and still end up calling mcount? (but I really doubt that is the
   case - but you never know).
 - Something else?

My thought was to crash the kernel as it is up and running and look at the
diassembled core to see what the instructions end up looking to get a further 
feel
for this.  But before I go with this are there some other ideas of what I 
should look
for?

Thanks!

Note: The "working on making the pte_flags be paravirt" patches are here:
http://darnok.org/results/baseline_pte_flags_pte_attrs/ if you are interested.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel