if_wi softc lock over 802.11 call (was: Re: LOR and other problems with -current.)

On Mon, 22 Mar 2004, Poul-Henning Kamp wrote:

> Mar 22 21:34:52 critter kernel: lock order reversal
> Mar 22 21:34:52 critter kernel: 1st 0xc4192bfc wi0 (network driver) _at_ dev/wi/if_wi.c:821
> Mar 22 21:34:52 critter kernel: 2nd 0xc40a0a7c radix node head (radix node head) _at_ net/if.c:590
> Mar 22 21:34:52 critter kernel: Stack backtrace:
> Mar 22 21:34:52 critter kernel: backtrace(0,ffffffff,c0693ea0,c06939a0,c065f8fc) at backtrace+0x12
> Mar 22 21:34:52 critter kernel: witness_checkorder(c40a0a7c,9,c0635b62,24e) at witness_checkorder+0x593
> Mar 22 21:34:52 critter kernel: _mtx_lock_flags(c40a0a7c,0,c0635b59,24e,d849bc30) at _mtx_lock_flags+0x67
> Mar 22 21:34:52 critter kernel: if_detach(c4192000,c4192000) at if_detach+0x35c
> Mar 22 21:34:52 critter kernel: ether_ifdetach(c4192000,c4192000,c41929c0,c4192000,c4192000) at ether_ifdetach+0x28
> Mar 22 21:34:52 critter kernel: ieee80211_ifdetach(c4192000,c4192000,c4192000,0,c4158d00) at ieee80211_ifdetach+0x31
> Mar 22 21:34:52 critter kernel: wi_detach(c4158d00) at wi_detach+0x61
> Mar 22 21:34:52 critter kernel: device_detach(c4158d00) at device_detach+0x56
> Mar 22 21:34:52 critter kernel: pccard_detach_card(c405a900) at pccard_detach_card+0x41
> Mar 22 21:34:52 critter kernel: exca_removal(c156b004,c156b000,30000016,d849bd18,c046de99) at exca_removal+0x46
> Mar 22 21:34:52 critter kernel: cbb_removal(c156b000,0,c4056bd0,c4055a50,c046de0c) at cbb_removal+0x1a
> Mar 22 21:34:52 critter kernel: cbb_event_thread(c156b000,d849bd48,c156b000,c046de0c,0) at cbb_event_thread+0x8d
> Mar 22 21:34:52 critter kernel: fork_exit(c046de0c,c156b000,d849bd48) at fork_exit+0xa8
> Mar 22 21:34:52 critter kernel: fork_trampoline() at fork_trampoline+0x8
> Mar 22 21:34:52 critter kernel: --- trap 0x1, eip = 0, esp = 0xd849bd7c, ebp = 0 ---

Looks like if_wi is holding the wi softc lock over a call to
ieee80211_ifdetach(), and that it probably shouldn't be doing that.

> Mar 22 21:37:35 critter kernel: lock order reversal
> Mar 22 21:37:35 critter kernel: 1st 0xc06b82c0 bpf global lock (bpf global lock) _at_ net/bpf.c:384
> Mar 22 21:37:35 critter kernel: 2nd 0xc4192bfc wi0 (network driver) _at_ dev/wi/if_wi.c:1085
> Mar 22 21:37:35 critter kernel: Stack backtrace:
> Mar 22 21:37:35 critter kernel: backtrace(0,ffffffff,c0694030,c0693ea0,c065f8fc) at backtrace+0x12
> Mar 22 21:37:35 critter kernel: witness_checkorder(c4192bfc,9,c062565a,43d) at witness_checkorder+0x593
> Mar 22 21:37:35 critter kernel: _mtx_lock_flags(c4192bfc,0,c0625651,43d) at _mtx_lock_flags+0x67
> Mar 22 21:37:35 critter kernel: wi_ioctl(c4192000,80206910,dc595a7c,c0635a4a,dc595aa0) at wi_ioctl+0x42
> Mar 22 21:37:35 critter kernel: ifpromisc(c4192000,0) at ifpromisc+0x92
> Mar 22 21:37:35 critter kernel: bpf_detachd(c4250000) at bpf_detachd+0x26
> Mar 22 21:37:35 critter kernel: bpfclose(c4251300,1,2000,c4197e70,c0668100) at bpfclose+0x7f
> Mar 22 21:37:35 critter kernel: spec_close(dc595b20,dc595b48,c052f06c,dc595b20,c067bb20) at spec_close+0x2b6
> Mar 22 21:37:35 critter kernel: spec_vnoperate(dc595b20) at spec_vnoperate+0x13
> Mar 22 21:37:35 critter kernel: vn_close(c44ab104,1,c4494b00,c4197e70,dc595bb8) at vn_close+0x40
> Mar 22 21:37:35 critter kernel: vn_closefile(c41c46e8,c4197e70) at vn_closefile+0x1e
> Mar 22 21:37:35 critter kernel: fdrop_locked(c41c46e8,c4197e70,c155f544,0,c062a474) at fdrop_locked+0x117
> Mar 22 21:37:35 critter kernel: fdrop(c41c46e8,c4197e70,3,c4197e70,dc595c14) at fdrop+0x24
> Mar 22 21:37:35 critter kernel: closef(c41c46e8,c4197e70) at closef+0x1db
> Mar 22 21:37:35 critter kernel: fdfree(c4197e70,c4285ed8,c4285dc0,0,0) at fdfree+0x2eb
> Mar 22 21:37:35 critter kernel: exit1(c4197e70,0,dc595d40,c05f0f8f,c4197e70) at exit1+0x45f
> Mar 22 21:37:35 critter kernel: exit1(c4197e70,dc595d14,1,9a,292) at exit1
> Mar 22 21:37:35 critter kernel: syscall(2f,2f,2f,813e000,ffffffff) at syscall+0x24b
> Mar 22 21:37:35 critter kernel: Xint0x80_syscall() at Xint0x80_syscall+0x1d
> Mar 22 21:37:35 critter kernel: --- syscall (1), eip = 0x28204ec3, esp = 0xbfbfe7bc, ebp = 0xbfbfe7d8 ---

This is a similar result, possibly from the same caus -- somewhere, if_wi
is holding the wi softc lock over a call into BPF, which it probably
shouldn't.  Later on, when the wi lock is grabbed while holding the bpf
lock (seems more reasonable), it turns up as a lock order reversal.  In
general, network device drivers should avoid holding their own locks while
calling into the network code, and most device drivers are careful to drop
their softc lock around ifp->if_input(), etc. 

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert_at_fledge.watson.org      Senior Research Scientist, McAfee Research