LOR tcp_input.c vs. tcp_usrreq.c (was: Re: 2 LORs on my NFS server.)

* Tilman Linneweh [Fr, 15 Aug 2003 at 16:17 GMT]:
> 
> My CURRENT is already a bit old:
> 
> # uname -a
> FreeBSD polly.arved.de 5.1-CURRENT FreeBSD 5.1-CURRENT #1: Sun Jul 20
> 01:00:14 CEST 2003    
> tilman_at_sauna.arved.de:/usr/obj/usr/src/CURRENT/sys/POLLY  i386

I updated my CURRENT to 

polly# uname -a
FreeBSD polly.arved.de 5.1-CURRENT FreeBSD 5.1-CURRENT #1: Sat Aug 16
10:11:52 CEST 2003    
tilman_at_sauna.arved.de:/usr/obj/usr/source/CURRENT/sys/POLLY  i386

and this LOR is reproducable. 
 
> This happend while the machine was NFS-serving around 3 clients with
> normal udp NFS and a  fourth. client tried to mount something via
> mount_nfs -T -a 2

The problem is the client with TCP mounts. I tried this time with a single
NetBSD client that does a TCP mount and cd'd to the mounted directory.

lock order reversal
 1st 0xc1a17278 inp (inp) _at_ /usr/source/CURRENT/sys/netinet/tcp_input.c:654
 2nd 0xc046bd6c tcp (tcp) _at_ /usr/source/CURRENT/sys/netinet/tcp_usrreq.c:621
Stack backtrace:
backtrace(1,0,ffffffff,c0445068,c04451d0) at backtrace+0x12
witness_lock(c046bd6c,8,c03c334c,26d,0) at witness_lock+0x55e
_mtx_lock_flags(c046bd6c,0,c03c334c,26d) at _mtx_lock_flags+0x7d
tcp_usr_rcvd(c1ce8800,80) at tcp_usr_rcvd+0x1b
soreceive(c1ce8800,c891ab1c,c891ab28,c891ab20,0) at soreceive+0x815
nfsrv_rcv(c1ce8800,c1a70780,4) at nfsrv_rcv+0x75
sowakeup(c1ce8800,c1ce884c) at sowakeup+0x7f
tcp_input(c0b9ac00,14) at tcp_input+0x11f6
ip_input(c0b9ac00) at ip_input+0x7c8
swi_net(0) at swi_net+0xe6
ithread_loop(c0b87180,c891ad48,c0b87180,c0221660,0) at ithread_loop+0x11c
fork_exit(c0221660,c0b87180,c891ad48) at fork_exit+0xab
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xc891ad7c, ebp = 0 ---
Debugger("witness_lock")
Stopped at      Debugger+0x45:  xchgl   %ebx,in_Debugger.0

#8  0xc0251271 in witness_lock (lock=0xc046bd6c, flags=8,
    file=0xc03c334c "/usr/source/CURRENT/sys/netinet/tcp_usrreq.c", line=621)
    at /usr/source/CURRENT/sys/kern/subr_witness.c:838
#9  0xc0229a7d in _mtx_lock_flags (m=0xc046bd6c, opts=0,
---Type <return> to continue, or q <return> to quit---
    file=0xc03c334c "/usr/source/CURRENT/sys/netinet/tcp_usrreq.c", line=621)
    at /usr/source/CURRENT/sys/kern/kern_mutex.c:336
#10 0xc02b951b in tcp_usr_rcvd (so=0x0, flags=128)
    at /usr/source/CURRENT/sys/netinet/tcp_usrreq.c:621
#11 0xc0266155 in soreceive (so=0xc1ce8800, psa=0xc891ab1c, uio=0xc891ab28,
    mp0=0xc891ab20, controlp=0x0, flagsp=0xc891ab24)
    at /usr/source/CURRENT/sys/kern/uipc_socket.c:1087
#12 0xc1a3efb5 in nfsrv_rcv (so=0xc1ce8800, arg=0xc1a70780, waitflag=4)
    at /usr/source/CURRENT/sys/nfsserver/nfs_srvsock.c:445
#13 0xc026783f in sowakeup (so=0xc1ce8800, sb=0xc1ce884c)
    at /usr/source/CURRENT/sys/kern/uipc_socket2.c:320
#14 0xc02b1336 in tcp_input (m=0xc0b9ac00, off0=20)
    at /usr/source/CURRENT/sys/netinet/tcp_input.c:1129
#15 0xc02abe08 in ip_input (m=0xc0b9ac00)
    at /usr/source/CURRENT/sys/netinet/ip_input.c:950
#16 0xc0293b06 in swi_net (dummy=0x0)