Re: r248583 Kernel panic: negative refcount 0xfffffe0031b59168

From: Pawel Jakub Dawidek <pjd_at_FreeBSD.org>
Date: Mon, 15 Apr 2013 10:35:17 +0200
On Sat, Apr 13, 2013 at 09:43:14PM -0700, Gleb Kurtsou wrote:
> On (22/03/2013 11:51), Shawn Webb wrote:
> > Hey All,
> > 
> > I'm not sure if this is a result of r248583 or a different commit, but I
> > hit a kernel panic when closing Chrome. I've linked to the info and
> > core.txt files below. If you need me to ship you the vmcore file, let me
> > know. It's 1.1GB in size.
> > 
> > Other than the pasted files, I'm not too sure where to go from here. If
> > there's any other info you need, please let me know. I'm a newb at
> > submitting this kind of stuff.
> > 
> > Paste of info file: http://ix.io/4Qo
> > Paste of core.txt file: http://ix.io/4Qp
> 
> Shawn, did you find workaround for the problem?
> 
> I've just upgraded to recent HEAD and see the same panic on closing
> chrome. Switching back to r247601 just before "Merge Capsicum overhaul"
> commit makes panic disappear.

I did receive Shawn's report some time ago, I even installed Chromium to
try to reproduce it, but it didn't crash for me yet.

If there are some easy, but reliable steps to reproduce it, like "open
this webpage in tab 1, then this webpage in tab 2, then close tab 1"
that would be great. This kernel coredump is not really useful, as we
this is legitimate case of decrementing reference counter. The problem
is that something decremented it earlier when it shouldn't or it wasn't
incremented somewhere. DTrace might be useful tool here if we could
instrument it to log backtrace of all increments and decrements done by
the Chromium processes.

> ~ # kgdb -n 1
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "amd64-marcel-freebsd"...
> 
> Unread portion of the kernel message buffer:
> VNASSERT failed
> 0xfffffe0196700760: tag none, type VBAD
>     usecount 0, writecount 0, refcount 0 mountedhere 0
>     flags (VV_NOSYNC|VI_DOOMED)
>     lock type zfs: UNLOCKED
> panic: No vop_advlock(0xfffffe0196700760, 0xffffff823adb9908)
> cpuid = 3
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffff823adb9740
> kdb_backtrace() at kdb_backtrace+0x39/frame 0xffffff823adb97f0
> vpanic() at vpanic+0x127/frame 0xffffff823adb9830
> kassert_panic() at kassert_panic+0x136/frame 0xffffff823adb98a0
> VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x92/frame 0xffffff823adb98d0
> closef() at closef+0x9a/frame 0xffffff823adb9960
> closefp() at closefp+0xa0/frame 0xffffff823adb99b0
> amd64_syscall() at amd64_syscall+0x1f9/frame 0xffffff823adb9ab0
> Xfast_syscall() at Xfast_syscall+0xfb/frame 0xffffff823adb9ab0
> --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x80aeaaa8a, rsp = 0x7ffffebf3f38, rbp = 0x7ffffebf3f50 ---
> [...]
> (kgdb) fr 0
> #0  doadump (textdump=1) at pcpu.h:231
> 231	pcpu.h: No such file or directory.
> 	in pcpu.h
> (kgdb) up
> #1  0xffffffff804f5827 in kern_reboot (howto=260) at /freebsd-src/local/sys/kern/kern_shutdown.c:447
> 447			doadump(TRUE);
> (kgdb) 
> #2  0xffffffff804f5d36 in vpanic (fmt=<value optimized out>, ap=<value optimized out>)
>     at /freebsd-src/local/sys/kern/kern_shutdown.c:754
> 754		kern_reboot(bootopt);
> (kgdb) 
> #3  0xffffffff804f5bc6 in kassert_panic (fmt=<value optimized out>)
>     at /freebsd-src/local/sys/kern/kern_shutdown.c:642
> 642			vpanic(fmt, ap);
> (kgdb) 
> #4  0xffffffff80747aa2 in VOP_ADVLOCK_APV (vop=<value optimized out>, a=0xffffff823adb9908)
>     at vnode_if.c:2522
> 2522		VNASSERT(vop != NULL, a->a_vp, ("No vop_advlock(%p, %p)", a->a_vp, a));
> (kgdb) 
> #5  0xffffffff804b8eaa in closef (fp=0xfffffe014da8ccd0, td=0xfffffe0014aea920) at vnode_if.h:1041
> 1041	vnode_if.h: No such file or directory.
> 	in vnode_if.h
> (kgdb) 
> #6  0xffffffff804b7030 in closefp (fdp=0xfffffe001c8c4800, fd=<value optimized out>, fp=0xfffffe014da8ccd0, 
>     td=0xfffffe0014aea920, holdleaders=<value optimized out>)
>     at /freebsd-src/local/sys/kern/kern_descrip.c:1136
> 1136		error = closef(fp, td);
> (kgdb) p *fp
> $5 = {f_data = 0xfffffe0196700760, f_ops = 0xffffffff80a477b8, f_cred = 0xfffffe0067907600, 
>   f_vnode = 0xfffffe0196700760, f_type = 1, f_vnread_flags = 0, f_flag = 3, f_count = 0, f_seqcount = 0, 
>   f_nextoff = 16388, f_vnun = {fvn_cdevpriv = 0x0, fvn_advice = 0x0}, f_offset = 16388, f_label = 0x0}
> (kgdb) p *fp
> $6 = {f_data = 0xfffffe0196700760, f_ops = 0xffffffff80a477b8, f_cred = 0xfffffe0067907600, 
>   f_vnode = 0xfffffe0196700760, f_type = 1, f_vnread_flags = 0, f_flag = 3, f_count = 0, f_seqcount = 0, 
>   f_nextoff = 16388, f_vnun = {fvn_cdevpriv = 0x0, fvn_advice = 0x0}, f_offset = 16388, f_label = 0x0}
> (kgdb) p fp->f_vnode
> $7 = (struct vnode *) 0xfffffe0196700760
> (kgdb) p *fp->f_vnode
> $8 = {v_tag = 0xffffffff807a3e35 "none", v_op = 0x0, v_data = 0x0, v_mount = 0x0, v_nmntvnodes = {
>     tqe_next = 0xfffffe014fd95760, tqe_prev = 0xfffffe011d500958}, v_un = {vu_mount = 0x0, vu_socket = 0x0, 
>     vu_cdev = 0x0, vu_fifoinfo = 0x0}, v_hashlist = {le_next = 0x0, le_prev = 0x0}, v_cache_src = {
>     lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, tqh_last = 0xfffffe01967007b0}, v_cache_dd = 0x0, 
>   v_lock = {lock_object = {lo_name = 0xffffffff80dddbb1 "zfs", lo_flags = 91881472, lo_data = 0, 
>       lo_witness = 0x0}, lk_lock = 1, lk_exslpfail = 0, lk_timo = 51, lk_pri = 96}, v_interlock = {
>     lock_object = {lo_name = 0xffffffff807bfbb9 "vnode interlock", lo_flags = 16908288, lo_data = 0, 
>       lo_witness = 0x0}, mtx_lock = 6}, v_vnlock = 0xfffffe01967007c8, v_actfreelist = {
>     tqe_next = 0xfffffe0031985b10, tqe_prev = 0xfffffe014fd95820}, v_bufobj = {bo_mtx = {lock_object = {
>         lo_name = 0xffffffff807bfbc9 "bufobj interlock", lo_flags = 16908288, lo_data = 0, 
>         lo_witness = 0x0}, mtx_lock = 6}, bo_ops = 0xffffffff80a5af10, bo_object = 0x0, bo_synclist = {
>       le_next = 0x0, le_prev = 0x0}, bo_private = 0xfffffe0196700760, __bo_vnode = 0xfffffe0196700760, 
>     bo_clean = {bv_hd = {tqh_first = 0x0, tqh_last = 0xfffffe0196700880}, bv_root = 0x0, bv_cnt = 0}, 
>     bo_dirty = {bv_hd = {tqh_first = 0x0, tqh_last = 0xfffffe01967008a0}, bv_root = 0x0, bv_cnt = 0}, 
>     bo_numoutput = 0, bo_flag = 0, bo_bsize = 131072}, v_pollinfo = 0x0, v_label = 0x0, v_lockf = 0x0, 
>   v_rl = {rl_waiters = {tqh_first = 0x0, tqh_last = 0xfffffe01967008e8}, rl_currdep = 0x0}, v_cstart = 0, 
>   v_lasta = 0, v_lastw = 0, v_clen = 0, v_holdcnt = 0, v_usecount = 0, v_iflag = 128, v_vflag = 4, 
>   v_writecount = 0, v_hash = 26636295, v_type = VBAD}
> 
> 
> # kgdb -n 0
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "amd64-marcel-freebsd"...
> 
> Unread portion of the kernel message buffer:
> panic: negative refcount 0xfffffe0059a400c8
> cpuid = 0
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffff823aff8770
> kdb_backtrace() at kdb_backtrace+0x39/frame 0xffffff823aff8820
> vpanic() at vpanic+0x127/frame 0xffffff823aff8860
> kassert_panic() at kassert_panic+0x136/frame 0xffffff823aff88d0
> closef() at closef+0x1ff/frame 0xffffff823aff8960
> closefp() at closefp+0xa0/frame 0xffffff823aff89b0
> amd64_syscall() at amd64_syscall+0x1f9/frame 0xffffff823aff8ab0
> Xfast_syscall() at Xfast_syscall+0xfb/frame 0xffffff823aff8ab0
> --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x80aeaaa8a, rsp = 0x7fffffffbd28, rbp = 0x7fffffffbd40 ---
> Uptime: 21m3s
> [...]
> (kgdb) bt
> #0  doadump (textdump=1) at pcpu.h:231
> #1  0xffffffff804f5827 in kern_reboot (howto=260) at /freebsd-src/local/sys/kern/kern_shutdown.c:447
> #2  0xffffffff804f5d36 in vpanic (fmt=<value optimized out>, ap=<value optimized out>)
>     at /freebsd-src/local/sys/kern/kern_shutdown.c:754
> #3  0xffffffff804f5bc6 in kassert_panic (fmt=<value optimized out>)
>     at /freebsd-src/local/sys/kern/kern_shutdown.c:642
> #4  0xffffffff804b900f in closef (fp=<value optimized out>, td=<value optimized out>) at refcount.h:66
> #5  0xffffffff804b7030 in closefp (fdp=0xfffffe018dc79800, fd=<value optimized out>, fp=0xfffffe0059a400a0, 
>     td=0xfffffe016dfca920, holdleaders=<value optimized out>)
>     at /freebsd-src/local/sys/kern/kern_descrip.c:1136
> #6  0xffffffff806e26c9 in amd64_syscall (td=0xfffffe016dfca920, traced=0) at subr_syscall.c:134
> #7  0xffffffff806cb13b in Xfast_syscall () at exception.S:387
> #8  0x000000080aeaaa8a in ?? ()
> Previous frame inner to this frame (corrupt stack?)
> Current language:  auto; currently minimal
> (kgdb) 
> 
> > 
> > Thanks,
> > 
> > Shawn Webb
> > _______________________________________________
> > freebsd-current_at_freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"

-- 
Pawel Jakub Dawidek                       http://www.wheelsystems.com
FreeBSD committer                         http://www.FreeBSD.org
Am I Evil? Yes, I Am!                     http://mobter.com

Received on Mon Apr 15 2013 - 06:33:17 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:36 UTC