Re: DTrace/FreeBSD source snapshot

John Birrell writes:
 > On Mon, Feb 04, 2008 at 04:34:12PM -0500, Andrew Gallatin wrote:
 > > 
 > > With this installed, I'm seeing a problem running any
 > > dtrace scripts:
 > > 
 > > dtrace: failed to compile script /nfs/home/gallatin/dtrace/printa.d:
 > > "/usr/lib/dtrace/psinfo.d", line 37: syntax error near "uid_t"
 > > 
 > > I'm guessing dtrace doesn't know what a uid_t is.   Is this some sort
 > > of installation problem on my part?
 > 
 > Hmmm. That looks like a CTF conversion problem, assuming that uid_t
 > is actually referenced in our kernel (which I think it should be).
 > 
 > Does a ctfdump of the kernel list uid_t?

There was no ctf data at all.  I build kernels the old
fashioned way in /sys (no buildkernel), and I ran config
prior to installworld, so the makefile did not have the CTF
Total pilot error on my part, sorry to have bothered you!
 > > panic: solaris assert: ((__curthread())->td_proc) == p, file:
 > 
 > Ah that's WIP. I shouldn't have added fasttrap to the dtraceall
 > module dependencies. Can you take it out or just load all the other
 > modules manually? You wo't get anywhere trying to trace userland stuff
 > yet.

That works, thanks!

 FWIW, I was mainly looking to try some of the kernel
profiling scripts that I use on Solaris and MacOSX.
They're variations on the example scripts.

When I ran the following on a fairly busy system:
#!/usr/sbin/dtrace -s

profile:::profile-997
{
        _at_a[stack(20)]=count();
}


END
{
        trunc(_at_a, 20);
        printa(_at_a);
}

I see nothing:

% ~/dtrace/profile_stack.d 
dtrace: script '/nfs/home/gallatin/dtrace/profile_stack.d' matched 2
probes
<20 seconds elapses>
^C
CPU     ID                    FUNCTION:NAME
  2      2                             :END 

            20064

Under similar load on Solaris (or MacOSX), I'd see a bunch of
different "hot" stacks.  A few more tries hung the system for ~30
seconds, until it crashed:

bge0: watchdog timeout -- resetting
spin lock 0xffffffff80a5ac20 (smp rendezvous) held by 0xffffff008c96aa50 (tid 100135) too long
panic: spin lock held too long
cpuid = 1
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
panic() at panic+0x17d
_mtx_lock_spin_failed() at _mtx_lock_spin_failed+0x39
_mtx_lock_spin() at _mtx_lock_spin+0x9e
smp_rendezvous_cpus() at smp_rendezvous_cpus+0xe1
dtrace_xcall() at dtrace_xcall+0x6a
dtrace_state_deadman() at dtrace_state_deadman+0x19
cyclic_clock() at cyclic_clock+0x12b
lapic_handle_timer() at lapic_handle_timer+0x8b
Xtimerint() at Xtimerint+0x67
--- interrupt, rip = 0xffffffff806fd9e6, rsp = 0xffffffffac289b90, rbp = 0xffffffffac289ba0 ---
acpi_cpu_c1() at acpi_cpu_c1+0x6
acpi_cpu_idle() at acpi_cpu_idle+0x19c
sched_idletd() at sched_idletd+0x46
fork_exit() at fork_exit+0x11f
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffffffac289d30, rbp = 0 ---


FWIW, this is an 8-way opteron.

Drew