Re: Panic and LOR on -CURRENT with ath

Andrea Campi wrote:
> Hi,
> 
> I have a Soekris box working as router/firewall/access point for my LAN, and
> after my latest update (a few days ago) I've been having a couple of issues:
> a LOR at boot, and a panic after some variable amount of time. I sort of
> remember seeing something similar to the panic on a mailing list, but can't
> find the thread.
> 
> Since they appear regularly, if there's anything I can do to help debug the
> issue I'm all ears.
> 
> Bye,
> 	Andrea
> 
> 
> lock order reversal: (sleepable after non-sleepable)
>  1st 0xc0d68d30 ath0 (network driver) _at_ /usr/src/sys/dev/ath/if_ath.c:4642
>  2nd 0xc0cfc878 user map (user map) _at_ /usr/src/sys/vm/vm_map.c:2997
> KDB: stack backtrace:
> witness_checkorder(c0cfc878,9,c05fed2f,bb5,c5b7c924) at witness_checkorder+0x383
> _sx_xlock(c0cfc878,c05fed2f,bb5,2b7c928,c5b7c924) at _sx_xlock+0x3c
> vm_map_lookup(c5b7c924,805e000,2,c5b7c928,c5b7c918,c5b7c91c,c5b7c8ff,c5b7c900) a
> t vm_map_lookup+0x30
> vm_fault(c0cfc834,805e000,2,8,c0d5e900) at vm_fault+0x63
> trap_pfault(805e000) at trap_pfault+0x10b
> trap(8,c0640028,c0d50028,805e000,c0da8e00) at trap+0x34a
> calltrap() at calltrap+0x5
> --- trap 0xc, eip = 0xc05c0316, esp = 0xc5b7ca2c, ebp = 0xc5b7ca60 ---
> generic_copyout(c0d65000,c0d90980,c0d68aac,c0286938,0) at generic_copyout+0x36
> ieee80211_ioctl(c0d681ac,c0286938,c0d90980) at ieee80211_ioctl+0x16a
> ath_ioctl(c0d65000,c0286938,c0d90980) at ath_ioctl+0xb2
> ifhwioctl(c0d90980,c0d5e900,c0d65000,c0fd7590,c0286938) at ifhwioctl+0x1fd
> ifioctl(c0fd7590,c0286938,c0d90980,c0d5e900) at ifioctl+0x5c
> soo_ioctl(c0de75e8,c0286938,c0d90980,c0ce6d80,c0d5e900) at soo_ioctl+0x29d
> ioctl(c0d5e900,c5b7cd04,3,13,206) at ioctl+0xfa
> syscall(3b,3b,3b,805d028,805d000) at syscall+0x110
> Xint0x80_syscall() at Xint0x80_syscall+0x1f

This is a driver lock held across an ioctl that does copyout and the 
destination faults.  I've tried to get people to buy into a solution to 
the general problem w/o luck.

> --- syscall (54, FreeBSD ELF32, ioctl), eip = 0x2813f69f, esp =fe5d8 ---
> 
> 
> 
> Fatal trap 12: page fault while in kernel mode
> fault virtual address   = 0xdeadc0de
> fault code              = supervisor read, page not present
> instruction pointer     = 0x20:0xc04dccc9
> stack pointer           = 0x28:0xc5705c10
> frame pointer           = 0x28:0xc5705c18
> code segment            = base 0x0, limit 0xfffff, type 0x1b
>                         = DPL 0, pres 1, def32 1, gran 1
> processor eflags        = interrupt enabled, resume, IOPL = 0
> current process         = 31 (swi6: task queue)
> [thread pid 31 tid 100018 ]
> Stopped at      m_tag_delete_chain+0x29:        movl    0(%ebx),%eax
> db> bt
> Tracing pid 31 tid 100018 td 0xc0cf3780
> m_tag_delete_chain(c103fb00,0,deadc0de,c5705c58,c057bc61) at m_tag_delete_chain+
> 0x29
> mb_dtor_mbuf(c103fb00,100,0) at mb_dtor_mbuf+0x38
> uma_zfree_arg(c0872e80,c103fb00,0) at uma_zfree_arg+0x2f1
> m_freem(c103fb00,c0d69188,0,c0fd2000,c5baa588) at m_freem+0x4e
> ath_tx_processq(1,c0d6940c,c5705ce8,c04c1e51,c0d68000) at ath_tx_processq+0x109
> ath_tx_proc_q0123(c0d68000,1,c0ce779c,0,c05eea3e) at ath_tx_proc_q0123+0x24
> taskqueue_run(c0ce7780,0,c0d0e830,0,c048ff00) at taskqueue_run+0x81
> ithread_loop(c0ce7700,c5705d38,c0ce7700,c048ff00,0) at ithread_loop+0x175
> fork_exit(c048ff00,c0ce7700,c5705d38) at fork_exit+0x83
> fork_trampoline() at fork_trampoline+0x8
> --- trap 0x1, eip = 0, esp = 0xc5705d6c, ebp = 0 ---

It appears the mbuf was reclaimed while sitting on the tx queue waiting 
to be reaped.  I've seen a few complaints of this sort but never enough 
info to start looking.  This problem seems to be common to soekris h/w 
or perhaps this general config which is common on soekris h/w.  Knowing 
the configuration might be useful; e.g. bridged config?  what packet 
filter package?  network setup?

	Sam