Re: 9.0-BETA3 lock order reversal in mount_smbfs

On Wed, 5 Oct 2011 19:39:56 -0700 Bob Finch wrote:

 BF> Attempting to mount a remote SMB share with mount_smbfs fails:

 BF> freebsd9b3# uname -a
 BF> FreeBSD freebsd9b3 9.0-BETA3 FreeBSD 9.0-BETA3 #0: Sat Sep 24 20:46:57 UTC 2011     root_at_obrian.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386
 BF> freebsd9b3# mount_smbfs -I smbhost -U xxx -W domain //smbhost/xxx /mnt
 BF> Password:
 BF> mount_smbfs: unable to open connection: syserr = No such file or directory

 BF> and displays the following kernel messages:

 BF> smb_co_lock: recursive lock for object 1
 BF> lock order reversal:
 BF> 1st 0xc2ef4608 smb_vc (smb_vc) _at_ /usr/src/sys/modules/smbfs/../../netsmb/smb_conn.c:325
 BF> 2nd 0xc2ffbc28 smbsm (smbsm) _at_ /usr/src/sys/modules/smbfs/../../netsmb/smb_conn.c:348
 BF> KDB: stack backtrace:
 BF> db_trace_self_wrapper(c0eff6ac,626d732f,2e2f7366,2e2e2f2e,74656e2f,...) at db_trace_self_wrapper+0x26
 BF> kdb_backtrace(c0a42bdb,c0f0300f,c29697b0,c29696e0,c76d298c,...) at kdb_backtrace+0x2a
 BF> _witness_debugger(c0f0300f,c2ffbc28,c2ff93df,c29696e0,c2ff9320,...) at _witness_debugger+0x25
 BF> witness_checkorder(c2ffbc28,9,c2ff9320,15c,c2ffbc48,...) at witness_checkorder+0x839
 BF> __lockmgr_args(c2ffbc28,80000,c2ffbc48,0,0,...) at __lockmgr_args+0x824
 BF> smb_co_lock(c2ffbc20,80000,2,2,c76d2b30,...) at smb_co_lock+0x73
 BF> smb_co_gone(c2ef4600,c76d2b88,c76d2b88,c76d2aac,c2ad5b00,...) at smb_co_gone+0x34
 BF> smb_sm_lookup(c76d2ad8,c76d2b14,c76d2b88,c76d2b30,c29f041c,...) at smb_sm_lookup+0xf0
 BF> smb_usr_lookup(c29f0400,c76d2b88,c76d2b94,c76d2b90,c76d2b7c,...) at smb_usr_lookup+0x98
 BF> nsmb_dev_ioctl(c2f76700,82fc6e6a,c29f0400,3,c2fce8a0,...) at nsmb_dev_ioctl+0x1d9
 BF> giant_ioctl(c2f76700,82fc6e6a,c29f0400,3,c2fce8a0,...) at giant_ioctl+0x75
 BF> devfs_ioctl_f(c2d417a8,82fc6e6a,c29f0400,c2c05e00,c2fce8a0,...) at devfs_ioctl_f+0x10b
 BF> kern_ioctl(c2fce8a0,3,82fc6e6a,c29f0400,6d2cec,...) at kern_ioctl+0x21d
 BF> sys_ioctl(c2fce8a0,c76d2cec,c0f493b6,c0eebb0e,246,...) at sys_ioctl+0x134
 BF> syscall(c76d2d28) at syscall+0x284
 BF> Xint0x80_syscall() at Xint0x80_syscall+0x21
 BF> --- syscall (54, FreeBSD ELF32, sys_ioctl), eip = 0x28193283, esp = 0xbfbfe35c, ebp = 0xbfbfe688 ---

The LOR appears after the problem (the connection gone) happened, on error
handling. So although it indicates that there is something wrong with our smb
locking this does not look like a cause of your issue.

 BF> Anything further I can do to help debug this problem?

Is this specific for 9.0-BETA3? Have you tried mounting the share with the
same parameters on another systems?

I think it could be useful to tcpdump the session and look at it in wireshark,
which understands the SMB protocol.

Also you might want to rebuild the kernel with this options in /etc/src.conf:

DEBUG_FLAGS=    -g -DSMB_SOCKET_DEBUG -DSMB_IOD_DEBUG -DNB_DEBUG -DSMB_VNODE_DEBUG

which will add many debugging messages. This might be helpful for troubleshooting.

-- 
Mikolaj Golub