ifconf() copies out holding mutex (was: Re: LOR)

On Sun, 5 Sep 2004, Hiroki Sato wrote:

>  I got the following LOR just after starting /etc/rc.d/sendmail
>  with -CURRENT as of Sep 4 on a SMP machine.  No panic occurred, but
>  this appears every time the system is booted.

The lock order reversal is a slightly obfuscated way to say "ifconf() 
tried to do a copyout() while holding a mutex".  This is a bug in 4.x
also, as copyout() processing a page fault can sleep, racing for access to
the interface and address lists with other kernel threads.  I guess the
'ifc->ifc_len' value can get pretty large, but the answer in the short
term may be to malloc in-kernel storage to use, or maybe map/pin the
pages.  Or, we could do something where we malloc kernel storage and
release the locks but perform some sort of safe "restart" to restart where
we left off (a sentinel/place holder/something).  This is hardly a high
performance interface, though -- getifaddrs(3) does a lot of work to
figure out how large to make that buffer.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert_at_fledge.watson.org      Principal Research Scientist, McAfee Research


> 
> lock order reversal
>  1st 0xc08ec200 ifnet (ifnet) _at_ /usr/src/sys/net/if.c:1489
>  2nd 0xc46703c8 user map (user map) _at_ /usr/src/sys/vm/vm_map.c:2994
> KDB: stack backtrace:
> kdb_backtrace(0,ffffffff,c08c7810,c08c8300,c08569ec) at kdb_backtrace+0x29
> witness_checkorder(c46703c8,9,c0810c9b,bb2) at witness_checkorder+0x544
> _sx_xlock(c46703c8,c0810c9b,bb2) at _sx_xlock+0x50
> _vm_map_lock_read(c4670384,c0810c9b,bb2,2000004,c4b715ac) at _vm_map_lock_read+0x37
> vm_map_lookup(e852ca84,813b000,2,e852ca88,e852ca78) at vm_map_lookup+0x28
> vm_fault(c4670384,813b000,2,8,c4adeb00) at vm_fault+0x66
> trap_pfault(e852cb4c,0,813b000) at trap_pfault+0xd2
> trap(e8520018,c0610010,c08e0010,813b000,e852cbc0) at trap+0x30d
> calltrap() at calltrap+0x5
> --- trap 0xc, eip = 0xc078d17a, esp = 0xe852cb8c, ebp = 0xe852cbec ---
> generic_copyout(c0086924,e852cc60,e852cc14,c061fa3a,c08c8968) at generic_copyout+0x36
> ifioctl(c4c7ca20,c0086924,e852cc60,c4adeb00,0) at ifioctl+0x25
> soo_ioctl(c4b4a8c4,c0086924,e852cc60,c4d5cd00,c4adeb00) at soo_ioctl+0x2b1
> ioctl(c4adeb00,e852cd14,3,2,282) at ioctl+0x3e0
> syscall(2f,2f,2f,0,0) at syscall+0x213
> Xint0x80_syscall() at Xint0x80_syscall+0x1f
> --- syscall (54, FreeBSD ELF32, ioctl), eip = 0x2828c22f, esp = 0xbfbfcd7c, ebp = 0xbfbfcf28 ---
> 
> -- 
> | Hiroki SATO
>