Re: IGMPv3 hot interface detach panics?

From: Paul B. Mahol <onemda_at_gmail.com>
Date: Thu, 12 Mar 2009 20:37:33 +0100
On 3/12/09, Bruce Simpson <bms_at_incunabulum.net> wrote:
> Weongyo Jeong wrote:
>> ...
>> This is one I have got from "Paul B. Mahol" <onemda_at_gmail.com>
>> yesterday and I think he might help you to get full backtrace:
>>
> Thanks for this.  I really do need a full backtrace to work out what to
> go on, however,
> as it looks like the compiler is inlining static functions here.
>
> Is IGMPv2 in use on this network? The stack will default to IGMPv3, it
> is possible that
> the call to igmp_ifdetach() needs to be forced to happen before
> in_purgemaddrs() causes
> the IP stack's reference to the in_multi to be freed.
>
> Are any multicast applications in use?
>
> Sam is seeing a very different condition with 802.11 VAPs, even though
> the symptoms are similar.
>
> I appreciate that it isn't always possible to test with all combinations
> of network drivers
> that people may be using, vlans vaps etc. and some of them do use parts
> of the network stack
> in different ways, i.e. stacking the use of otherwise refcounted
> structures, etc.
>
> This looks like it is coming from USB2 and NDIS in some way. I am not at
> all familiar with
> those subsystems and don't use them, so help from others would be very
> appreciated,
> however I will try my best to help fix, the root cause may be more general.

Here is one with usb2 and if_rum,
(panic on detach happens only if inet address was asigned to wlan0:
ifconfig wlan0 inet x.x.x.x)

db:0:kdb.enter.unknown>  run lockinfo
db:1:lockinfo> show locks
db:1:locks>  show alllocks
Process 733 (usbus4) thread 0xc43fc000 (100077)
db:1:alllocks>  show lockedvnods
Locked vnodes
db:0:kdb.enter.unknown>  show pcpu
cpuid        = 0
curthread    = 0xc43fc000: pid 733 "usbus4"
curpcb       = 0xc3bafd90
fpcurthread  = none
idlethread   = 0xc3d09d20: pid 10 "idle: cpu0"
APIC ID      = 0
currentldt   = 0x50
spin locks held:
db:0:kdb.enter.unknown>  bt
Tracing pid 733 tid 100077 td 0xc43fc000
in_ifdetach(c3de9800,c3de9a30,32b,c3bafbc4,c4469ab1,...) at in_ifdetach+0x18d
if_detach(c3de9800,0,c44a5d2c,237,c3de9800,...) at if_detach+0xfd
rum_detach(c445b580,4,c06213f6,9e7,c04ce719,...) at rum_detach+0xab
device_detach(c445b580,c42c9faa,c3f8d5e0,6,2,...) at device_detach+0x8c
usb2_detach_device(c44bd000,ff,1,10,c061d235,...) at usb2_detach_device+0x16a
uhub_explore(c4402000,0,c42c9267,d8,c4446d34,...) at uhub_explore+0x1ab
usb2_bus_explore(c4446d34,0,c42d1f13,51,c068fdc0,...) at usb2_bus_explore+0xb9
usb2_process(c4446c70,c3bafd38,c061a98c,32d,c42967ec,...) at usb2_process+0xda
fork_exit(c42bb3a0,c4446c70,c3bafd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xc3bafd70, ebp = 0 ---

ugen4.2: <Ralink> at usbus4
rum0: <Ralink 802.11 bg WLAN, class 0/0, rev 2.00/0.01, addr 2> on usbus4
rum0: MAC/BBP RT2573 (rev 0x2573a), RF RT2528
rum0: need multicast update callback
rum0: at uhub4, port 6, addr 2 (disconnected)
Kernel page fault with the following non-sleepable locks held:
exclusive sleep mutex if_addr_mtx (if_addr_mtx) r = 0 (0xc3de9a40)
locked _at_ /usr/local/src/sys/netinet/in.c:1041
exclusive sleep mutex in_multi_mtx (in_multi_mtx) r = 0 (0xc07f9174)
locked _at_ /usr/local/src/sys/netinet/in.c:1033
KDB: stack backtrace:
db_trace_self_wrapper(c0621b7e,c3bafa64,c04e5995,c062e32e,409,...) at
db_trace_self_wrapper+0x26
kdb_backtrace(c062e32e,409,ffffffff,c07cb19c,c3bafa9c,...) at kdb_backtrace+0x29
_witness_debugger(c0623edd,c3bafab0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c06407ba,c3c8da90,c43fc000,...) at witness_warn+0x1fd
trap(c3bafb3c) at trap+0x153
calltrap() at calltrap+0x6
--- trap 0xc, eip = 0xc055474d, esp = 0xc3bafb7c, ebp = 0xc3bafb9c ---
in_ifdetach(c3de9800,c3de9a30,32b,c3bafbc4,c4469ab1,...) at in_ifdetach+0x18d
if_detach(c3de9800,0,c44a5d2c,237,c3de9800,...) at if_detach+0xfd
rum_detach(c445b580,4,c06213f6,9e7,c04ce719,...) at rum_detach+0xab
device_detach(c445b580,c42c9faa,c3f8d5e0,6,2,...) at device_detach+0x8c
usb2_detach_device(c44bd000,ff,1,10,c061d235,...) at usb2_detach_device+0x16a
uhub_explore(c4402000,0,c42c9267,d8,c4446d34,...) at uhub_explore+0x1ab
usb2_bus_explore(c4446d34,0,c42d1f13,51,c068fdc0,...) at usb2_bus_explore+0xb9
usb2_process(c4446c70,c3bafd38,c061a98c,32d,c42967ec,...) at usb2_process+0xda
fork_exit(c42bb3a0,c4446c70,c3bafd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xc3bafd70, ebp = 0 ---


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x0
fault code              = supervisor write, page not present
instruction pointer     = 0x20:0xc055474d
stack pointer           = 0x28:0xc3bafb7c
frame pointer           = 0x28:0xc3bafb9c
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 733 (usbus4)
exclusive sleep mutex if_addr_mtx (if_addr_mtx) r = 0 (0xc3de9a40)
locked _at_ /usr/local/src/sys/netinet/in.c:1041
exclusive sleep mutex in_multi_mtx (in_multi_mtx) r = 0 (0xc07f9174)
locked _at_ /usr/local/src/sys/netinet/in.c:1033
exclusive sleep mutex Giant (Giant) r = 0 (0xc068b810) locked _at_
/usr/local/src/sys/modules/usb/usb/../../../dev/usb/controller/usb_controller.c:216
exclusive sx 123456789ABCDEF - USB config SX lock (123456789ABCDEF -
USB config SX lock) r = 0 (0xc44bd03c) locked _at_
/usr/local/src/sys/modules/usb/usb/../../../dev/usb/usb_device.c:941
exclusive sleep mutex if_addr_mtx (if_addr_mtx) r = 0 (0xc3de9a40)
locked _at_ /usr/local/src/sys/netinet/in.c:1041
exclusive sleep mutex in_multi_mtx (in_multi_mtx) r = 0 (0xc07f9174)
locked _at_ /usr/local/src/sys/netinet/in.c:1033
exclusive sleep mutex Giant (Giant) r = 0 (0xc068b810) locked _at_
/usr/local/src/sys/modules/usb/usb/../../../dev/usb/controller/usb_controller.c:216
exclusive sx 123456789ABCDEF - USB config SX lock (123456789ABCDEF -
USB config SX lock) r = 0 (0xc44bd03c) locked _at_
/usr/local/src/sys/modules/usb/usb/../../../dev/usb/usb_device.c:941


(gdb) l *in_ifdetach+0x18d
0xc055474d is in in_ifdetach (/usr/local/src/sys/netinet/in.c:1046).
1041            IF_ADDR_LOCK(ifp);
1042            TAILQ_FOREACH(ifma, &ifp->if_multiaddrs, ifma_link) {
1043                    if (ifma->ifma_addr->sa_family != AF_INET)
1044                            continue;
1045                    inm = (struct in_multi *)ifma->ifma_protospec;
1046                    LIST_INSERT_HEAD(&purgeinms, inm, inm_link);
1047            }
1048            IF_ADDR_UNLOCK(ifp);
1049
1050            LIST_FOREACH_SAFE(inm, &purgeinms, inm_link, tinm) {
(gdb) l *if_detach+0xfd
0xc054285d is in if_detach (/usr/local/src/sys/net/if.c:847).
842              * routes are expected to be removed by the
IPv6-specific kernel API.
843              * Otherwise, the kernel will detect some
inconsistency and bark it.
844              */
845             in6_ifdetach(ifp);
846     #endif
847             if_purgemaddrs(ifp);
848
849             /*
850              * Remove link ifaddr pointer and maybe decrement if_index.
851              * Clean up all addresses.
(gdb) l *rum_detach+0xab
0x27fb is in rum_detach
(/usr/local/src/sys/modules/usb/rum/../../../dev/usb/wlan/if_rum.c:573).
568
569             if (ifp) {
570                     ic = ifp->if_l2com;
571                     bpfdetach(ifp);
572                     ieee80211_ifdetach(ic);
573                     if_free(ifp);
574             }
575             cv_destroy(&sc->sc_cmd_cv);
576             mtx_destroy(&sc->sc_mtx);
577


Here is same crash, but via kgdb:

(kgdb) bt
#0  doadump () at pcpu.h:246
#1  0xc04a744e in boot (howto=260) at
/usr/local/src/sys/kern/kern_shutdown.c:420
#2  0xc04a7722 in panic (fmt=Variable "fmt" is not available.
) at /usr/local/src/sys/kern/kern_shutdown.c:576
#3  0xc05f8ba3 in trap_fatal (frame=0xc3bdfb3c, eva=0)
    at /usr/local/src/sys/i386/i386/trap.c:926
#4  0xc05f9441 in trap (frame=0xc3bdfb3c) at
/usr/local/src/sys/i386/i386/trap.c:318
#5  0xc05dfd9b in calltrap () at /usr/local/src/sys/i386/i386/exception.s:165
#6  0xc055474d in in_ifdetach (ifp=0xc3fc9c00) at
/usr/local/src/sys/netinet/in.c:1017
#7  0xc054285d in if_detach (ifp=0xc3fc9c00) at /usr/local/src/sys/net/if.c:835
#8  0xc45087fb in rum_detach (self=0xc44e6d80)
    at /usr/local/src/sys/modules/usb/rum/../../../dev/usb/wlan/if_rum.c:572
#9  0xc04ceb5c in device_detach (dev=0xc44e6d80) at device_if.h:212
#10 0xc434998a in usb2_detach_device (udev=0xc44b2000, iface_index=32 ' ',
    free_subdev=1 '\001')
    at /usr/local/src/sys/modules/usb/usb/../../../dev/usb/usb_device.c:901
#11 0xc4351d2b in uhub_explore (udev=0xc4156000)
    at /usr/local/src/sys/modules/usb/usb/../../../dev/usb/usb_hub.c:301
#12 0xc4342fb9 in usb2_bus_explore (pm=0xc43fbd34)
    at /usr/local/src/sys/modules/usb/usb/../../../dev/usb/controller/usb_controller.c:227
#13 0xc435447a in usb2_process (arg=0xc43fbc70)
    at /usr/local/src/sys/modules/usb/usb/../../../dev/usb/usb_process.c:139
#14 0xc0484458 in fork_exit (callout=0xc43543a0 <usb2_process>, arg=0xc43fbc70,
    frame=0xc3bdfd38) at /usr/local/src/sys/kern/kern_fork.c:821
#15 0xc05dfe10 in fork_trampoline () at
/usr/local/src/sys/i386/i386/exception.s:270
(kgdb) frame 6
#6  0xc055474d in in_ifdetach (ifp=0xc3fc9c00) at
/usr/local/src/sys/netinet/in.c:1017
1017    }
(kgdb) frame 5
#5  0xc05dfd9b in calltrap () at /usr/local/src/sys/i386/i386/exception.s:165
165             call    trap
Current language:  auto; currently asm
(kgdb) frame 7
#7  0xc054285d in if_detach (ifp=0xc3fc9c00) at /usr/local/src/sys/net/if.c:835
835             in_ifdetach(ifp);
Current language:  auto; currently c


--
Paul
Received on Thu Mar 12 2009 - 18:37:35 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:43 UTC