Re: LOR:vfs_bio.c and ufs_dirhash.c

Ivailo Bonev wrote:
> I've installed 8-CURRENT on my HP laptop, snapshot from pub.allbsd.org with HEAD sources from yesterday, but on first startup I see LOR. Everything works ok, though...
> Here is log from messages:

<snip>

> Jun 15 17:46:53 ibb kernel: lock order reversal:
> Jun 15 17:46:53 ibb kernel: 1st 0xd9537bb0 bufwait (bufwait) _at_ /usr/src/sys/kern/vfs_bio.c:2558
> Jun 15 17:46:53 ibb kernel: 2nd 0xc5efd000 dirhash (dirhash) _at_ /usr/src/sys/ufs/ufs/ufs_dirhash.c:285
> Jun 15 17:46:53 ibb kernel: KDB: stack backtrace:
> Jun 15 17:46:53 ibb kernel: db_trace_self_wrapper(c0c5e1c8,e813a87c,c08b15d5,c08a250b,c0c6100b,...) at db_trace_self_wrapper+0x26
> Jun 15 17:46:53 ibb kernel: kdb_backtrace(c08a250b,c0c6100b,c552acf0,c5530c00,e813a8d8,...) at kdb_backtrace+0x29
> Jun 15 17:46:53 ibb kernel: _witness_debugger(c0c6100b,c5efd000,c0c810bd,c5530c00,c0c80d56,...) at _witness_debugger+0x25
> Jun 15 17:46:53 ibb kernel: witness_checkorder(c5efd000,9,c0c80d56,11d,0,...) at witness_checkorder+0x839
> Jun 15 17:46:53 ibb kernel: _sx_xlock(c5efd000,0,c0c80d56,11d,c6015c3c,...) at _sx_xlock+0x85
> Jun 15 17:46:53 ibb kernel: ufsdirhash_acquire(d9537b50,da6db800,200,da6db81c,e813a9a8,...) at ufsdirhash_acquire+0x35
> Jun 15 17:46:53 ibb kernel: ufsdirhash_add(c6015c3c,e813aa20,81c,e813a994,e813a998,...) at ufsdirhash_add+0x13
> Jun 15 17:46:53 ibb kernel: ufs_direnter(c5ff596c,c6139218,e813aa20,e813ac04,d956c200,...) at ufs_direnter+0x729
> Jun 15 17:46:53 ibb kernel: ufs_mkdir(e813ac28,ead,0,0,e813ab70,...) at ufs_mkdir+0x897
> Jun 15 17:46:53 ibb kernel: VOP_MKDIR_APV(c0d5eec0,e813ac28,e813ac04,e813ab70,0,...) at VOP_MKDIR_APV+0xa5
> Jun 15 17:46:53 ibb kernel: kern_mkdirat(c5cda480,ffffff9c,28528f80,0,1ed,...) at kern_mkdirat+0x268
> Jun 15 17:46:53 ibb kernel: kern_mkdir(c5cda480,28528f80,0,1ed,e813ad2c,...) at kern_mkdir+0x2e
> Jun 15 17:46:53 ibb kernel: mkdir(c5cda480,e813acf8,8,c0c5e284,c0d3f240,...) at mkdir+0x29
> Jun 15 17:46:53 ibb kernel: syscall(e813ad38) at syscall+0x2a3
> Jun 15 17:46:53 ibb kernel: Xint0x80_syscall() at Xint0x80_syscall+0x20
> Jun 15 17:46:53 ibb kernel: --- syscall (136, FreeBSD ELF32, mkdir), eip = 0x283063c3, esp = 0xbf4f9d1c, ebp = 0xbf4f9d48 ---
> 
> Mail me, if I can help with anything else...

This is known, and shouldn't be anything to worry about because it will 
not result in a deadlock. See the comment added in r187474:

+  * WITNESS reports a lock order reversal between the "bufwait" lock
+  * and the "dirhash" lock.  However, this specific reversal will not
+  * cause a deadlock.  To get a deadlock, one would have to lock a
+  * buffer followed by the dirhash while a second thread locked a
+  * buffer while holding the dirhash lock.  The second order can happen
+  * under a shared or exclusive vnode lock for the associated directory
+  * in lookup().  The first order, however, can only happen under an
+  * exclusive vnode lock (e.g. unlink(), rename(), etc.).  Thus, for
+  * a thread to be doing a "bufwait" -> "dirhash" order, it has to hold
+  * an exclusive vnode lock.  That exclusive vnode lock will prevent
+  * any other threads from doing a "dirhash" -> "bufwait" order.

See also http://sources.zabbadoz.net/freebsd/lor/261.html but note that 
you are seeing different line numbers in vfs_bio.c and ufs_dirhash.c 
because those files have been changed since this page was first created 
back in September.

Nick