Re: panic: lockmgr still held [tmpfs] [vm_map_remove()->vdropl()] (r262186: Thu Feb 20)

On 3/3/2014 12:06 PM, John Baldwin wrote:
> On Sunday, March 02, 2014 10:58:45 am Bryan Drewery wrote:
>> On 2/28/2014 3:18 PM, John Baldwin wrote:
>>> On Friday, February 28, 2014 9:18:51 am Bryan Drewery wrote:
>>>> While using poudriere:
>>>>
>>>>> Unread portion of the kernel message buffer:
>>>>> panic: lockmgr still held
>>>>> cpuid = 12
>>>>> KDB: stack backtrace:
>>>>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe124804f7a0
>>>>> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe124804f850
>>>>> vpanic() at vpanic+0x126/frame 0xfffffe124804f890
>>>>> kassert_panic() at kassert_panic+0x139/frame 0xfffffe124804f900
>>>>> lockdestroy() at lockdestroy+0x3b/frame 0xfffffe124804f920
>>>>> vdropl() at vdropl+0x1c8/frame 0xfffffe124804f960
>>>>> vm_object_deallocate() at vm_object_deallocate+0x10b/frame 0xfffffe124804f9c0
>>>>> vm_map_process_deferred() at vm_map_process_deferred+0x89/frame 0xfffffe124804f9f0
>>>>> vm_map_remove() at vm_map_remove+0xc8/frame 0xfffffe124804fa20
>>>>> vmspace_exit() at vmspace_exit+0xc9/frame 0xfffffe124804fa60
>>>>> exit1() at exit1+0x541/frame 0xfffffe124804fad0
>>>>> sys_sys_exit() at sys_sys_exit+0xe/frame 0xfffffe124804fae0
>>>>> ia32_syscall() at ia32_syscall+0x270/frame 0xfffffe124804fbf0
>>>>> Xint0x80_syscall() at Xint0x80_syscall+0x95/frame 0xfffffe124804fbf0
>>>>> --- syscall (1, FreeBSD ELF32, sys_sys_exit), rip = 0x281014df, rsp = 0xffffc45c, rbp = 0xffffc468 ---
>>>>
>>>>> #4  0xffffffff808c00db in lockdestroy (lk=0xfffff80a88a285f0) at /usr/src/sys/kern/kern_lock.c:440
>>>>> 440             KASSERT(lk->lk_lock == LK_UNLOCKED, ("lockmgr still held"));
>>>>> (kgdb) print *lk
>>>>> $1 = {lock_object = {lo_name = 0xffffffff8201a1bd "tmpfs", lo_flags = 116588552, lo_data = 0, lo_witness = 0xfffffe00006fec00}, lk_lock = 
>>> 18446735288132049184, lk_exslpfail = 0,
>>>>>   lk_timo = 51, lk_pri = 96}
>>>
>>> Can you please grab people.freebsd.org/~jhb/gdb/*
>>>
>>> and then do 'cd /path/to/files', 'source gdb6', 'frame 4', 'lockmgr_owner lk'?
>>>
>>
>> (kgdb) lockmgr_owner lk
>> td: 0xfffff80272c61920
>> pid: 55040, p_comm: testprog
> 
> Can you get a stack trace of that program?  ('proc 55040', 'bt')
> 

It's the same as original:

> Loaded symbols for /boot/kernel/sem.ko.symbols
> #0  doadump (textdump=1) at pcpu.h:219
> 219             __asm("movq %%gs:%1,%0" : "=r" (td)
> (kgdb) bt
> #0  doadump (textdump=1) at pcpu.h:219
> #1  0xffffffff808e33a7 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:452
> #2  0xffffffff808e38b5 in vpanic (fmt=<value optimized out>, ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:759
> #3  0xffffffff808e3749 in kassert_panic (fmt=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:647
> #4  0xffffffff808c00db in lockdestroy (lk=0xfffff80a88a285f0) at /usr/src/sys/kern/kern_lock.c:440
> #5  0xffffffff80990d38 in vdropl (vp=0xfffff80a88a28588) at /usr/src/sys/kern/vfs_subr.c:2417
> #6  0xffffffff80b73cbb in vm_object_deallocate (object=<value optimized out>) at /usr/src/sys/vm/vm_object.c:539
> #7  0xffffffff80b6a3a9 in vm_map_process_deferred () at /usr/src/sys/vm/vm_map.c:2763
> #8  0xffffffff80b6e9c8 in vm_map_remove (map=<value optimized out>, start=<value optimized out>, end=<value optimized out>) at /usr/src/sys/vm/vm_map.c:488
> #9  0xffffffff80b6a179 in vmspace_exit (td=0xfffff80272c61920) at /usr/src/sys/vm/vm_map.c:327
> #10 0xffffffff808a7411 in exit1 (td=0xfffff80272c61920, rv=<value optimized out>) at /usr/src/sys/kern/kern_exit.c:319
> #11 0xffffffff808a6ece in sys_sys_exit (td=0x0, uap=<value optimized out>) at /usr/src/sys/kern/kern_exit.c:119
> #12 0xffffffff80dd2720 in ia32_syscall (frame=0xfffffe124804fc00) at subr_syscall.c:133
> #13 0xffffffff80cd6325 in Xint0x80_syscall () at ia32_exception.S:73
> #14 0x00000000281014df in ?? ()
> Previous frame inner to this frame (corrupt stack?)
> Current language:  auto; currently minimal
> (kgdb) proc 55040
> [Switching to thread 668 (Thread 100898)]#0  doadump (textdump=1) at pcpu.h:219
> 219             __asm("movq %%gs:%1,%0" : "=r" (td)
> (kgdb) bt
> #0  doadump (textdump=1) at pcpu.h:219
> #1  0xffffffff808e33a7 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:452
> #2  0xffffffff808e38b5 in vpanic (fmt=<value optimized out>, ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:759
> #3  0xffffffff808e3749 in kassert_panic (fmt=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:647
> #4  0xffffffff808c00db in lockdestroy (lk=0xfffff80a88a285f0) at /usr/src/sys/kern/kern_lock.c:440
> #5  0xffffffff80990d38 in vdropl (vp=0xfffff80a88a28588) at /usr/src/sys/kern/vfs_subr.c:2417
> #6  0xffffffff80b73cbb in vm_object_deallocate (object=<value optimized out>) at /usr/src/sys/vm/vm_object.c:539
> #7  0xffffffff80b6a3a9 in vm_map_process_deferred () at /usr/src/sys/vm/vm_map.c:2763
> #8  0xffffffff80b6e9c8 in vm_map_remove (map=<value optimized out>, start=<value optimized out>, end=<value optimized out>) at /usr/src/sys/vm/vm_map.c:488
> #9  0xffffffff80b6a179 in vmspace_exit (td=0xfffff80272c61920) at /usr/src/sys/vm/vm_map.c:327
> #10 0xffffffff808a7411 in exit1 (td=0xfffff80272c61920, rv=<value optimized out>) at /usr/src/sys/kern/kern_exit.c:319
> #11 0xffffffff808a6ece in sys_sys_exit (td=0x0, uap=<value optimized out>) at /usr/src/sys/kern/kern_exit.c:119
> #12 0xffffffff80dd2720 in ia32_syscall (frame=0xfffffe124804fc00) at subr_syscall.c:133
> #13 0xffffffff80cd6325 in Xint0x80_syscall () at ia32_exception.S:73
> #14 0x00000000281014df in ?? ()
> (kgdb)



-- 
Regards,
Bryan Drewery