Re: New DTrace source snapshot

On Feb 16, 2008 11:42 PM, John Birrell <jb_at_what-creek.com> wrote:
> On Sun, Feb 17, 2008 at 12:33:57AM +0200, Andrew Pogrebennyk wrote:
> > Snapshot dtrace-20080211.tar.bz compiled well for me on i386. However
> > during boot with DTrace I am getting the following tracebacks:
> >
> > GEOM_LABEL: Label for provider acd0 is iso9660/FreeBSD_Install.
> > WARNING: WITNESS option enabled, expect reduced performance.
> > lock order reversal:
>
> The lock order reversal warnings are a fault with current at the
> moment. If I had my way the cause would be backed out until there
> is a solution. I understand that it is a witness improvment that
> has caused the warnings to start appearing.... meaning that the
> change has exposed an existing problem that wasn't warned obut
> before.
>
> They certainly aren't DTrace change specific.
>
> > sh*:::line
> > {
> >         _at_lines[pid, uid, copyinstr(arg0)] = count();
> > }
>
> There is no sh provider yet.
>
> --
> John Birrell
>

I've had several panics while running Bonnie64 benchmars on ZFS on the
Dtrace snapshot.
I'm not entirely sure if this is Dtrace or ZFS related (the backtrace
shows dtrace functions),
but here is what textdump shows  anyways :

vertsion : FreeBSD 8.0-CURRENT #3: Sun Feb 17 16:38:29 EET 2008
    root_at_tester69:/usr/obj/usr/src/sys/GENERIC***

*** This is GENERIC from the DTrace snapshot with INVARIANTS and
WITNESS turned off

panic message : spin lock held too long

msgbuf :
spin lock 0xffffffff80aad760 (smp rendezvous) held by
0xffffff000b02d6e0 (tid 100080) too long
panic: spin lock held too long
cpuid = 1
KDB: enter: panic

db:0:kdb.enter.panic>  show allpcpu
Current CPU: 1

cpuid        = 0
curthread    = 0xffffff000b486370: pid 141 "spa_zio_intr_2"
curpcb       = 0xffffffffd83e6d40
fpcurthread  = none
idlethread   = 0xffffff0002234370: pid 11 "idle: cpu0"

cpuid        = 1
curthread    = 0xffffff0002dde6e0: pid 34 "arc_reclaim_thread"
curpcb       = 0xffffffffd5f99d40
fpcurthread  = none
idlethread   = 0xffffff00022346e0: pid 11 "idle: cpu1"

db:0:kdb.enter.panic>  bt
Tracing pid 34 tid 100051 td 0xffffff0002dde6e0
kdb_enter() at kdb_enter+0x3d
panic() at panic+0x176
_mtx_lock_spin_failed() at _mtx_lock_spin_failed+0x39
_mtx_lock_spin() at _mtx_lock_spin+0x9e
smp_rendezvous_cpus() at smp_rendezvous_cpus+0xe1
dtrace_xcall() at dtrace_xcall+0x6a
dtrace_dynvar_clean() at dtrace_dynvar_clean+0xbe
dtrace_state_clean() at dtrace_state_clean+0x29
cyclic_clock() at cyclic_clock+0x12b
lapic_handle_timer() at lapic_handle_timer+0x8b
Xtimerint() at Xtimerint+0x67
--- interrupt, rip = 0xffffffff80c444e5, rsp = 0xffffffffd5f99b60, rbp
= 0xffffffffd5f99b80 ---
arc_do_user_evicts() at arc_do_user_evicts+0x85
arc_reclaim_thread() at arc_reclaim_thread+0x1db
fork_exit() at fork_exit+0x11f
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffffffd5f99d30, rbp = 0 ---

Full textdump (debug.ddb.scripting.scripts="textdump set; capture on;
show pcpu; bt; show locks; ps; alltrace; show lockedvnods; show
alllocks; capture off; call doadump; reset") here :
http://bg.freebsd.org/~ndenev/crashdumps/textdump_of_dtrace_or_zfs_panic.tar

  --Niki