Performance Tools in FreeBSD

Dear all:

      I wonder to know which kind of performance tools FreeBSD hackers use
to identify scalability or performance

      bottlenecks. I really have a hard time in that. To identify
bottlenecks in our benchmarks, i try PMC and Dtrace.

      However, PMC seems not to be supported in AMD Opteron CPU because i
get a "invalid arguments" error when

      i use the command pmcstat -S instructions -O <outputfile>. However,
"instructions, cycles,..." are documented in

     pmc manual. By the way, I enable the options in configuration file
options HWPMC_HOOKS and device hwpmc.

     So i give up PMC and turn to Dtrace in BSD. Dtrace in FreeBSD can work,
but the information is meaningless.

     Profile provider cannot be used. (When i used, the kernel goes into
kdb.) Tick provider can be used (It can only work

      on one CPU by definition). However, when i use aggregation like
"dtrace -n 'tick-10ms /arg0!=0/{_at_ks[probefunc] = count();}'"

      There are no human readable results. Only tools i can use is the
stack() command. However, the results are limited.

      I know Dtrace is a tool in FreeBSD under development.

      Are there any tools recommended to do kernel profiling or callgraph
effectively? (which functions take the most time)



                                           Best Wishes!



                                                Yan