Re: panic in arptimer in r289937

From: Adrian Chadd <adrian_at_freebsd.org>
Date: Sat, 31 Oct 2015 09:46:03 -0400
On 31 October 2015 at 09:34, Alexander V. Chernikov
<melifaro_at_freebsd.org> wrote:
>
>
> 31.10.2015, 05:32, "Adrian Chadd" <adrian_at_freebsd.org>:
>> Hiya,
>>
>> Here's a panic from arptimer:
> Hi Adrian,
>
> As far as I see, line 205 in if_ether.c is IF_AFDATA_LOCK(ifp) which happens after LLE_WUNLOCK().
> So, it looks like (pre-cached) ifp had been freed before locking ifdata.
> Do you have any more details on that? (e.g. was some interface detached at that moment, is it reproducible, etc..)
>
> From a quick glance, potential use-after-free has been possible for quite a long time, but I wonder why it hasn't been observed before.
> Probably lltable_free() changes might have triggered that.
>
> I'll take a deeper look on that and reply.

Hiya!

Thanks for your quick response.

I mean, I use wifi, and ARPs can get lost / transmit can get delayed /
etc. I'm also testing through a MIPS CPU based bridge, so I'm also not
bridging at line rate. (The above is from one of the x86 laptops doing
the traffic test.) These are both reasons why I may be poking at a
path that you don't normally see. :)

I appreciate you taking a very quick look at this!

Thanks,



-adrian

>
>>
>> (kgdb) bt
>> #0 doadump (textdump=0) at pcpu.h:221
>> #1 0xffffffff803666b6 in db_fncall (dummy1=<value optimized out>,
>> dummy2=<value optimized out>, dummy3=<value optimized out>,
>> dummy4=<value optimized out>) at
>> /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:568
>> #2 0xffffffff8036614e in db_command (cmd_table=0x0) at
>> /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:440
>> #3 0xffffffff80365ee4 in db_command_loop () at
>> /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:493
>> #4 0xffffffff8036897b in db_trap (type=<value optimized out>, code=0)
>> at /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_main.c:251
>> #5 0xffffffff8096c0f3 in kdb_trap (type=9, code=0, tf=<value
>> optimized out>) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/subr_kdb.c:654
>> #6 0xffffffff80d34c81 in trap_fatal (frame=0xfffffe022815d7a0,
>> eva=<value optimized out>) at
>> /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/trap.c:829
>> #7 0xffffffff80d34951 in trap (frame=<value optimized out>) at
>> /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/trap.c:203
>> #8 0xffffffff80d149f7 in calltrap () at
>> /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/exception.S:234
>> #9 0xffffffff8092c3fb in _rw_wlock_cookie (c=0xdeadc0dedeadc2de,
>> file=0xffffffff81211b1f
>> "/usr/home/adrian/work/freebsd/head/src/sys/netinet/if_ether.c",
>> line=205)
>>     at /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_rwlock.c:261
>> #10 0xffffffff80a2487f in arptimer (arg=0xfffff8005ecc4000) at
>> /usr/home/adrian/work/freebsd/head/src/sys/netinet/if_ether.c:205
>> #11 0xffffffff80944c24 in softclock_call_cc (c=0xfffff8005ecc40a8,
>> cc=0xffffffff81b2d480, direct=0) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_timeout.c:722
>> #12 0xffffffff80944f87 in softclock (arg=<value optimized out>) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_timeout.c:851
>> #13 0xffffffff808f7eb6 in intr_event_execute_handlers (p=<value
>> optimized out>, ie=0xfffff800035a6600) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_intr.c:1262
>> #14 0xffffffff808f8546 in ithread_loop (arg=0xfffff800032c47c0) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_intr.c:1275
>> #15 0xffffffff808f57a4 in fork_exit (callout=0xffffffff808f84a0
>> <ithread_loop>, arg=0xfffff800032c47c0, frame=0xfffffe022815dac0) at
>> /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_fork.c:1011
>> #16 0xffffffff80d14f2e in fork_trampoline () at
>> /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/exception.S:609
>> #17 0x0000000000000000 in ?? ()
>> Current language: auto; currently minimal
>>
>> (kgdb) print *(struct llentry *)c_arg
>> $2 = {lle_next = {le_next = 0x0, le_prev = 0xfffff8005e867dc8},
>> r_l3addr = {addr4 = {s_addr = 16782508}, addr6 = {__u6_addr =
>> {__u6_addr8 = 0xfffff8005ecc4010 "�\024", __u6_addr16 =
>> 0xfffff8005ecc4010,
>>         __u6_addr32 = 0xfffff8005ecc4010}}}, ll_addr = {mac_aligned =
>> 110869256150596, mac16 = 0xfffff8005ecc4020, mac8 = 0xfffff8005ecc4020
>> "D\036���d"}, spare0 = 0, spare1 = 0, lle_tbl = 0xfffff8005e867e00,
>>   lle_head = 0xfffff8005e867dc8, lle_free = 0xffffffff80a2c5d0
>> <in_lltable_destroy_lle>, la_hold = 0x0, la_numheld = 0, la_expire =
>> 2110, la_flags = 1, la_asked = 0, la_preempt = 5, ln_state = 0,
>> ln_router = 0, ln_ntick = 0,
>>   lle_refcnt = 1, lle_chain = {le_next = 0x0, le_prev = 0x0},
>> lle_timer = {c_links = {le = {le_next = 0x0, le_prev =
>> 0xffffffff81b2d588}, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0,
>> tqe_prev = 0xffffffff81b2d588}},
>>     c_time = 9066299815445, c_precision = 322122525000, c_arg =
>> 0xfffff8005ecc4000, c_func = 0xffffffff80a246e0 <arptimer>, c_lock =
>> 0x0, c_flags = 0, c_iflags = 144, c_cpu = 0}, lle_lock = {lock_object
>> = {
>>       lo_name = 0xffffffff8120fbce "lle", lo_flags = 90374144, lo_data
>> = 0, lo_witness = 0xfffffe0000b53c80}, rw_lock = 1}}
>>
>> ..
>>
>> (kgdb) print *((struct llentry *)c_arg)->lle_tbl
>> $4 = {llt_link = {sle_next = 0xdeadc0dedeadc0de}, llt_af = -559038242,
>> llt_hsize = -559038242, lle_head = 0xdeadc0dedeadc0de, llt_ifp =
>> 0xdeadc0dedeadc0de, llt_lookup = 0xdeadc0dedeadc0de,
>>   llt_alloc_entry = 0xdeadc0dedeadc0de, llt_delete_entry =
>> 0xdeadc0dedeadc0de, llt_prefix_free = 0xdeadc0dedeadc0de,
>> llt_dump_entry = 0xdeadc0dedeadc0de, llt_hash = 0xdeadc0dedeadc0de,
>> llt_match_prefix = 0xdeadc0dedeadc0de,
>>   llt_free_entry = 0xdeadc0dedeadc0de, llt_foreach_entry =
>> 0xdeadc0dedeadc0de, llt_link_entry = 0xdeadc0dedeadc0de,
>> llt_unlink_entry = 0xdeadc0dedeadc0de, llt_fill_sa_entry =
>> 0xdeadc0dedeadc0de,
>>   llt_free_tbl = 0xdeadc0dedeadc0de}
>>
>> :(
>>
>> Any ideas on where next to look?
>>
>> -adrian
>> _______________________________________________
>> freebsd-net_at_freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe_at_freebsd.org"
Received on Sat Oct 31 2015 - 12:46:04 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:00 UTC