Re: panic in zfs on reboot

On 11/6/15 1:12 PM, Kurt Lidl wrote:
> I updated my machine that tracks head and rebooted it,
> and it panic'd during the 'shutdown -r' execution.
>
>    Panic String: solaris assert: zrl->zr_refcount == 0 (0x2 == 0x0),
> file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zrlock.c,
> line: 64
>
> (kgdb) bt
> #0  doadump (textdump=1) at pcpu.h:221
> #1  0xffffffff80708ec5 in kern_reboot (howto=260)
>      at /usr/src/sys/kern/kern_shutdown.c:364
> #2  0xffffffff8070949b in vpanic (fmt=<value optimized out>,
>      ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:757
> #3  0xffffffff807094e3 in panic (fmt=0x0)
>      at /usr/src/sys/kern/kern_shutdown.c:688
> #4  0xffffffff81b3c25f in assfail3 (a=<value optimized out>,
>      lv=<value optimized out>, op=<value optimized out>,
>      rv=<value optimized out>, f=<value optimized out>, l=<value
> optimized out>)
>      at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:91
> #5  0xffffffff8187fe54 in zrl_destroy (zrl=0xfffff8001d429820)
>      at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zrlock.c:64
> #6  0xffffffff81804432 in dnode_special_close (dnh=0xfffff8001d429820)
>      at
> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c:1005
> #7  0xffffffff817fa608 in dmu_objset_evict_done (os=0xfffff8001d429800)
>      at
> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:726
> #8  0xffffffff8180f191 in dsl_dataset_evict (dbu=0xfffff8000f957000)
>      at
> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dataset.c:287
> #9  0xffffffff80756ae0 in taskqueue_run_locked (queue=0xfffff8000d00cd00)
>      at /usr/src/sys/kern/subr_taskqueue.c:430
> #10 0xffffffff807575f8 in taskqueue_thread_loop (arg=<value optimized out>)
>      at /usr/src/sys/kern/subr_taskqueue.c:683
> #11 0xffffffff806cf6b4 in fork_exit (
>      callout=0xffffffff80757570 <taskqueue_thread_loop>,
>      arg=0xfffff8000d01e020, frame=0xfffffe085c058ac0)
>      at /usr/src/sys/kern/kern_fork.c:1011
> #12 0xffffffff809ae70e in fork_trampoline ()
>      at /usr/src/sys/amd64/amd64/exception.S:609
> #13 0x0000000000000000 in ?? ()
>
> I have a vmcore for this panic.  I have saved off the installed kernel
> directory for future examination.
>
> -Kurt

I should have copied in the whole info.2 file:

Dump header from device: /dev/gpt/swap0
   Architecture: amd64
   Architecture Version: 2
   Dump Length: 1404231680
   Blocksize: 512
   Dumptime: Fri Nov  6 12:10:25 2015
   Hostname: busybox.pix.net
   Magic: FreeBSD Kernel Dump
   Version String: FreeBSD 11.0-CURRENT #86 r290447M: Fri Nov  6 
11:00:18 EST 2015
     lidl_at_busybox.pix.net:/usr/obj/usr/src/sys/BUSYBOX
   Panic String: solaris assert: zrl->zr_refcount == 0 (0x2 == 0x0), 
file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zrlock.c, 
line: 64
   Dump Parity: 375889703
   Bounds: 2
   Dump Status: good

The only delta in my source tree on this machine is a patch
to the pagezero.S file, which I've been running for several months,
so I know it's not that.

(https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199151 for the
delta, if you care.  I need to benchmark this more fully, but just
haven't gotten around to doing it yet.)

-Kurt