Re: LOR with filedesc structure and Giant

On Sun, 27 Jul 2003, Kris Kennaway wrote:

> After upgrading last night, one of the package machines found this:

I've bumped into some similar problems -- it's a property of how we
current lock select().  We hold the file descriptor lock for the duration
of polling each object being "selected", and if any of those objects has
to grab a lock for any reason, it has to implicitly fall after the file
descriptor lock.  I actually run into this in some of our MAC code,
because I need to grab a vnode lock to authorize polling the vnode using
VOP_POLL(), and since the vnode lock is a sleep lock, this generates a
WITNESS warning.  Unfortunately, it's not immediately clear what a better
locking scheme would look like without going overboard on the fine-grained
side.  We probably need to grab Giant before entering the select code
since it's highly likely something in there will require Giant -- it
reaches down into VFS, the device stuff, socket code, tc.

> lock order reversal
>  1st 0xc6c1c334 filedesc structure (filedesc structure) _at_
> /a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:902
>  2nd 0xc04aa120 Giant (Giant) _at_
> /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:372
> Stack backtrace:
> backtrace(c043d4af,c04aa120,c0439aa4,c0439aa4,c0434e3d) at backtrace+0x17
> witness_lock(c04aa120,8,c0434e3d,174,1bc) at witness_lock+0x672
> _mtx_lock_flags(c04aa120,0,c0434e3d,174,c043daba) at _mtx_lock_flags+0xba
> spec_poll(d8dddaf8,d8dddb18,c02d119c,d8dddaf8,c04939a0) at spec_poll+0x134
> spec_vnoperate(d8dddaf8,c04939a0,c520b124,40,c675e300) at spec_vnoperate+0x18
> vn_poll(c44c5e14,40,c675e300,c6222d10,c675e300) at vn_poll+0x3c
> selscan(c6222d10,d8dddb98,d8dddb88,6,4) at selscan+0x13e
> kern_select(c6222d10,6,bfbff5c0,0,0) at kern_select+0x36f
> select(c6222d10,d8dddd10,c0455899,3ee,5) at select+0x66
> syscall(2f,2f,2f,8055050,bfbff5b8) at syscall+0x273
> Xint0x80_syscall() at Xint0x80_syscall+0x1d
> --- syscall (93), eip = 0x280ccacc, esp = 0x2832eb68, ebp = 0x2832ebc0 ---
> Debugger("witness_lock")
> Stopped at      Debugger+0x54:  xchgl   %ebx,in_Debugger.0
> 
> Kris