Re: LOR route vr0

On Sat, 27 Aug 2005, M. Warner Losh wrote:

> In message: <Pine.BSF.4.53.0508270912550.969_at_e0-0.zab2.int.zabbadoz.net>
>            "Bjoern A. Zeeb" <bzeeb-lists_at_lists.zabbadoz.net> writes:
> : > lock order reversal
> : >  1st 0xc17621ec rtentry (rtentry) _at_ /usr/src/sys/net/route.c:1269
> : >  2nd 0xc15ec938 vr0 (network driver) _at_ /usr/src/sys/pci/if_vr.c:1391
> :
> : added with ID 140: http://sources.zabbadoz.net/freebsd/lor.html#140
>
> I've noticed a *HUGE* number of LORs that look like this:
>
> ock order reversal
> 1st 0xc17490e4 rtentry (rtentry) _at_ sys/netinet/if_ether.c:445
> 2nd 0xc15c94b0 rl1 (network driver) _at_ sys/pci/if_rl.c:1451

Generally speaking, network interface device driver locks follow network 
stack locks in the lock order.  However, I've not really looked much at 
the route table locking so can't speak to whether that is the case 
specifically for routing locks.  If it is, the below traces reflect the 
correct order, and you might want to add a hard-coded entry to witness in 
order to catch the reverse order.  Lock order reversals between the 
network stack and device drivers tend to occur as a result of the device 
driver calling into the network stack while holding the device driver 
mutex.  Someone (tm) should work out if the right order is route locks -> 
device driver locks, as it's likely a common calss of bugs across many 
drivers.

Robert N M Watson

> KDB: stack backtrace:
> kdb_backtrace(c07dcab1,c15c94b0,c160a1b0,c07c54d7,c07f401e) at kdb_backtrace+0x2e
> witness_checkorder(c15c94b0,9,c07f401e,5ab,c07e32bd) at witness_checkorder+0x6c3
> _mtx_lock_flags(c15c94b0,0,c07f401e,5ab,c152cc00) at _mtx_lock_flags+0x8a
> rl_start(c152cc00,1,c07e2eae,836) at rl_start+0x37
> if_start(c152cc00,0,c07e32bd,195,202) at if_start+0x99
> ether_output_frame(c152cc00,c169c100,6,c0562c28,c169c100)
> 	 at ether_output_frame+0x218
> ether_output(c152cc00,c169c100,cbfe79f0,0,2,c1740001,2302,c07e66e8,1bd,519)
> 	at ether_output+0x47e
> arprequest(c152cc00,c16cfcc8,cbfe7ae4,c15fa6ab,c05998a6) at arprequest+0x109
> arpresolve(c152cc00,c1749084,c169a400,cbfe7ae0,cbfe7a64) at arpresolve+0x32d
> ether_output(c152cc00,c169a400,cbfe7ae0,c1749084,0) at ether_output+0x7b
> ip_output(c169a400,0,cbfe7adc,0,0) at ip_output+0xb7a
>
> and am seeing the following in my newly locked ed driver:
>
> lock order reversal
> 1st 0xc1cb3588 rtentry (rtentry) _at_ net/route.c:1269
> 2nd 0xc1fd3420 ed1 (network driver) _at_ /dell/imp/p4/newcard/src/sys/modules/ed/../../dev/ed/if_ed.c:697
> KDB: stack backtrace:
> kdb_backtrace(0,ffffffff,c0680950,c067f5a0,c064bd44) at kdb_backtrace+0x29
> witness_checkorder(c1fd3420,9,c201ff8b,2b9) at witness_checkorder+0x52c
> _mtx_lock_flags(c1fd3420,0,c201ff8b,2b9,c1a86c00) at _mtx_lock_flags+0x5b
> ed_start(c1a86c00) at ed_start+0x1f
> if_start(c1a86c00) at if_start+0x7b
> ether_output_frame(c1a86c00,c1bbeb00,c04c0920,ffffffff,0) at ether_output_frame+0x1dc
> ether_output(c1a86c00,c1bbeb00,e5832a38,0,2) at ether_output+0x3e4
> arprequest(c1a86c00,c1d77ac8,e5832b08,c20236ab) at arprequest+0xd8
> arpresolve(c1a86c00,c1cb3528,c1bbed00,e5832b04,e5832aa8) at arpresolve+0x30b
> ether_output(c1a86c00,c1bbed00,e5832b04,c1cb3528,c1d77a00) at ether_output+0x6b
> ip_output(c1bbed00,0,e5832b00,0,0) at ip_output+0x78c
> udp_output(c1cb1168,c1bbed00,0,0,c1a8d600) at udp_output+0x4a7
> udp_send(c1d59c84,0,c1bbed00,0,0) at udp_send+0x1a
> sosend(c1d59c84,0,e5832c3c,c1bbed00,0) at sosend+0x5e3
> kern_sendit(c1a8d600,4,e5832cbc,0,0) at kern_sendit+0x104
> sendit(c1a8d600,4,e5832cbc,0,807a023) at sendit+0x163
> sendto(c1a8d600,e5832d04,6,0,206) at sendto+0x4d
> syscall(3b,3b,3b,805a000,28219fa4) at syscall+0x22f
>
> and was wondering if there's a common cause.
>
> Warner
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>