Re: UFS Witness LoR + 5 other LoRs

Am Mon, 26 Jan 2009 14:01:03 -0800
schrieb Sean Bruno <sean.bruno_at_dsl-only.net>:

> Anyone else seeing this LoR today?
> 
> lock order reversal:
>  1st 0xc456b044 user map (user map)
> _at_ /root/bsd/head/sys/vm/vm_map.c:3198
>  2nd 0xc4911ad0 ufs (ufs) _at_ /root/bsd/head/sys/kern/vfs_subr.c:2071
> KDB: stack backtrace:
> db_trace_self_wrapper(c0bf0913,c417990c,c087a4b5,c0c1458c,0,...) at
> db_trace_self_wrapper+0x26
> kdb_backtrace(c0c1458c,0,c4521728,c4526730,c4179968,...) at
> kdb_backtrace+0x29
> _witness_debugger(c0bf3680,c4911ad0,c0be6d88,c4526730,c0bfa429,...) at
> _witness_debugger+0x25
> witness_checkorder(c4911ad0,1,c0bfa429,817,0,...) at
> witness_checkorder +0x839
> __lockmgr_args(c4911ad0,200501,c4911aec,0,0,...) at
> __lockmgr_args+0x237
> ffs_lock(c4179a78,c087a25b,c0c1697a,200501,c4911a78,...) at ffs_lock
> +0x8a VOP_LOCK1_APV(c0cf7ca0,c4179a78,c4567e24,c0d0bdc0,c4911a78,...)
> at VOP_LOCK1_APV+0xb5
> _vn_lock(c4911a78,200501,c0bfa429,817,4,...) at _vn_lock+0x5e
> vget(c4911a78,200501,c4567d80,4b4,0,...) at vget+0xc9
> vnode_pager_lock(c187c1f0,0,c0c13f07,127,c4179c18,...) at
> vnode_pager_lock+0x1e0
> vm_fault(c456b000,80db000,2,8,80db620,...) at vm_fault+0x1df
> trap_pfault(5,0,c0c241d1,2e7,c4565d34,...) at trap_pfault+0x118
> trap(c4179d38) at trap+0x289
> calltrap() at calltrap+0x6
> --- trap 0xc, eip = 0x80480e5, esp = 0xbfbfeef0, ebp = 0xbfbfef10 ---

Hi,

I have similar LORs. One of them might be affecting Gnome2. I cannot
even save anything once the LOR happened. nautilus is also hanging
around. I cannot even kill it. This occurs since I migrated
my /usr/home to ZFS (see fourth, fifth LoR).

Here one like yours:

lock order reversal:
 1st 0xffffff0002402070 user map (user map)
_at_ /usr/src/sys/vm/vm_map.c:3198 2nd 0xffffff0002b8a7f8 ufs (ufs)
_at_ /usr/src/sys/kern/vfs_subr.c:2071
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x2e
witness_checkorder() at witness_checkorder+0x81e
__lockmgr_args() at __lockmgr_args+0xca8
ffs_lock() at ffs_lock+0x8c
VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b
_vn_lock() at _vn_lock+0x47
vget() at vget+0x8b
vnode_pager_lock() at vnode_pager_lock+0x1d0
vm_fault() at vm_fault+0x1e2
trap_pfault() at trap_pfault+0x128
trap() at trap+0x51c
calltrap() at calltrap+0x8
--- trap 0xc, rip = 0x40014f, rsp = 0x7fffffffee70, rbp =
0x7fffffffee90 ---


Here a second one:

lock order reversal:
 1st 0xfffffffe803eb610 bufwait (bufwait)
_at_ /usr/src/sys/kern/vfs_bio.c:2443 2nd 0xffffff00080d2000 dirhash
(dirhash) _at_ /usr/src/sys/ufs/ufs/ufs_dirhash.c:275
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x2e
witness_checkorder() at witness_checkorder+0x81e
_sx_xlock() at _sx_xlock+0x55
ufsdirhash_acquire() at ufsdirhash_acquire+0x33
ufsdirhash_add() at ufsdirhash_add+0x19
ufs_direnter() at ufs_direnter+0x88b
ufs_mkdir() at ufs_mkdir+0x633
VOP_MKDIR_APV() at VOP_MKDIR_APV+0x93
kern_mkdirat() at kern_mkdirat+0x2b1
syscall() at syscall+0x1bf
Xfast_syscall() at Xfast_syscall+0xab
--- syscall (136, FreeBSD ELF64, mkdir), rip = 0x800719e9c, rsp =
0x7fffffffed08, rbp = 0x7fffffffef76 ---


Here the third:

lock order reversal:
 1st 0xffffff0008648248 filedesc structure (filedesc structure)
_at_ /usr/src/sys/kern/kern_descrip.c:1076 2nd 0xffffff0019110ba8 ufs
(ufs) _at_ /usr/src/sys/kern/vfs_subr.c:4057
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x2e
witness_checkorder() at witness_checkorder+0x81e
__lockmgr_args() at __lockmgr_args+0xc2a
ffs_lock() at ffs_lock+0x8c
VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b
_vn_lock() at _vn_lock+0x47
knlist_remove_kq() at knlist_remove_kq+0x73
knote_fdclose() at knote_fdclose+0x177
kern_close() at kern_close+0xe6
syscall() at syscall+0x1bf
Xfast_syscall() at Xfast_syscall+0xab
--- syscall (6, FreeBSD ELF64, close), rip = 0x800e35e8c, rsp =
0x7fffffffe678, rbp = 0x801063a00 ---



And thes two are really nasty (ZFS):

lock order reversal:
 1st 0xffffff00195ed620 zfs (zfs) _at_ /usr/src/sys/kern/vfs_vnops.c:531
 2nd 0xffffff00080ab360 user map (user map)
_at_ /usr/src/sys/vm/vm_map.c:3198 KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x2e
witness_checkorder() at witness_checkorder+0x81e
_sx_slock() at _sx_slock+0x55
vm_map_lookup() at vm_map_lookup+0x47
vm_fault() at vm_fault+0xfe
trap_pfault() at trap_pfault+0x128
trap() at trap+0x347
calltrap() at calltrap+0x8
--- trap 0xc, rip = 0xffffffff8081ed4b, rsp = 0xfffffffe97261850, rbp =
0xfffffffe972618d0 --- copyout() at copyout+0x3b
dmu_read_uio() at dmu_read_uio+0x98
zfs_freebsd_read() at zfs_freebsd_read+0x56d
vn_read() at vn_read+0x267
dofileread() at dofileread+0xa1
kern_readv() at kern_readv+0x60
read() at read+0x54
syscall() at syscall+0x1bf
Xfast_syscall() at Xfast_syscall+0xab
--- syscall (3, FreeBSD ELF64, read), rip = 0x80647becc, rsp =
0x7ffffffed8c8, rbp = 0x806682f80 ---

lock order reversal:
 1st 0xffffff0019bc2270 zfs (zfs) _at_ /usr/src/sys/kern/vfs_subr.c:2071
 2nd 0xffffff0019a77448 filedesc structure (filedesc structure)
_at_ /usr/src/sys/kern/vfs_syscalls.c:1720
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x2e
witness_checkorder() at witness_checkorder+0x81e
_sx_slock() at _sx_slock+0x55
kern_symlinkat() at kern_symlinkat+0x1f8
syscall() at syscall+0x1bf
Xfast_syscall() at Xfast_syscall+0xab
--- syscall (57, FreeBSD ELF64, symlink), rip = 0x8049cb13c, rsp =
0x7fffffffd6f8, rbp = 0x7fffffffdd10 ---

And one with NFS:

lock order reversal:
 1st 0xffffff00080a6940 user map (user map)
_at_ /usr/src/sys/vm/vm_map.c:3198 2nd 0xffffff00566a9ba8 nfs (nfs)
_at_ /usr/src/sys/kern/vfs_subr.c:2071
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x2e
witness_checkorder() at witness_checkorder+0x81e
__lockmgr_args() at __lockmgr_args+0xca8
vop_stdlock() at vop_stdlock+0x39
VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b
_vn_lock() at _vn_lock+0x47
vget() at vget+0x8b
vnode_pager_lock() at vnode_pager_lock+0x1d0
vm_fault() at vm_fault+0x1e2
trap_pfault() at trap_pfault+0x128
trap() at trap+0x347
calltrap() at calltrap+0x8
--- trap 0xc, rip = 0xffffffff8081edcd, rsp = 0xfffffffe972b67d0, rbp =
0xfffffffe972b6850 --- copyin() at copyin+0x3d
ffs_write() at ffs_write+0x2f8
VOP_WRITE_APV() at VOP_WRITE_APV+0xfe
vn_write() at vn_write+0x23f
dofilewrite() at dofilewrite+0x85
kern_writev() at kern_writev+0x60
write() at write+0x54
syscall() at syscall+0x1bf
Xfast_syscall() at Xfast_syscall+0xab
--- syscall (4, FreeBSD ELF64, write), rip = 0x80072beac, rsp =
0x7fffffffe5d8, rbp = 0x800537000 ---


Can you beat my 6 LoRs? ;)


FreeBSD zelda.local 8.0-CURRENT FreeBSD 8.0-CURRENT #0: Thu Jan 29
22:15:16 CET 2009     root_at_zelda.local:/usr/obj/usr/src/sys/ZELDA  amd64

--
Martin