Re: panic in arptimer in r289937

From: Alexander V. Chernikov <melifaro_at_freebsd.org>
Date: Sat, 31 Oct 2015 18:01:15 +0300
31.10.2015, 16:46, "Adrian Chadd" <adrian_at_freebsd.org>:
> On 31 October 2015 at 09:34, Alexander V. Chernikov
> <melifaro_at_freebsd.org> wrote:
>>  31.10.2015, 05:32, "Adrian Chadd" <adrian_at_freebsd.org>:
>>>  Hiya,
>>>
>>>  Here's a panic from arptimer:
>>  Hi Adrian,
>>
>>  As far as I see, line 205 in if_ether.c is IF_AFDATA_LOCK(ifp) which happens after LLE_WUNLOCK().
>>  So, it looks like (pre-cached) ifp had been freed before locking ifdata.
>>  Do you have any more details on that? (e.g. was some interface detached at that moment, is it reproducible, etc..)
>>
>>  From a quick glance, potential use-after-free has been possible for quite a long time, but I wonder why it hasn't been observed before.
>>  Probably lltable_free() changes might have triggered that.
>>
>>  I'll take a deeper look on that and reply.
>
> Hiya!
>
> Thanks for your quick response.
>
> I mean, I use wifi, and ARPs can get lost / transmit can get delayed /
> etc. I'm also testing through a MIPS CPU based bridge, so I'm also not
I remember that :)
> bridging at line rate. (The above is from one of the x86 laptops doing
> the traffic test.) These are both reasons why I may be poking at a
> path that you don't normally see. :)
Yup. So, once again, could you provide a bit more details? :)
Was it related with any interface being destroyed?
What was the test scenario (just bridging between interfaces?)
Is this reproducible?
>
> I appreciate you taking a very quick look at this!
>
> Thanks,
>
> -adrian
>
>>>  (kgdb) bt
>>>  #0 doadump (textdump=0) at pcpu.h:221
>>>  #1 0xffffffff803666b6 in db_fncall (dummy1=<value optimized out>,
>>>  dummy2=<value optimized out>, dummy3=<value optimized out>,
>>>  dummy4=<value optimized out>) at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:568
>>>  #2 0xffffffff8036614e in db_command (cmd_table=0x0) at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:440
>>>  #3 0xffffffff80365ee4 in db_command_loop () at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_command.c:493
>>>  #4 0xffffffff8036897b in db_trap (type=<value optimized out>, code=0)
>>>  at /usr/home/adrian/work/freebsd/head/src/sys/ddb/db_main.c:251
>>>  #5 0xffffffff8096c0f3 in kdb_trap (type=9, code=0, tf=<value
>>>  optimized out>) at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/subr_kdb.c:654
>>>  #6 0xffffffff80d34c81 in trap_fatal (frame=0xfffffe022815d7a0,
>>>  eva=<value optimized out>) at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/trap.c:829
>>>  #7 0xffffffff80d34951 in trap (frame=<value optimized out>) at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/trap.c:203
>>>  #8 0xffffffff80d149f7 in calltrap () at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/exception.S:234
>>>  #9 0xffffffff8092c3fb in _rw_wlock_cookie (c=0xdeadc0dedeadc2de,
>>>  file=0xffffffff81211b1f
>>>  "/usr/home/adrian/work/freebsd/head/src/sys/netinet/if_ether.c",
>>>  line=205)
>>>      at /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_rwlock.c:261
>>>  #10 0xffffffff80a2487f in arptimer (arg=0xfffff8005ecc4000) at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/netinet/if_ether.c:205
>>>  #11 0xffffffff80944c24 in softclock_call_cc (c=0xfffff8005ecc40a8,
>>>  cc=0xffffffff81b2d480, direct=0) at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_timeout.c:722
>>>  #12 0xffffffff80944f87 in softclock (arg=<value optimized out>) at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_timeout.c:851
>>>  #13 0xffffffff808f7eb6 in intr_event_execute_handlers (p=<value
>>>  optimized out>, ie=0xfffff800035a6600) at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_intr.c:1262
>>>  #14 0xffffffff808f8546 in ithread_loop (arg=0xfffff800032c47c0) at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_intr.c:1275
>>>  #15 0xffffffff808f57a4 in fork_exit (callout=0xffffffff808f84a0
>>>  <ithread_loop>, arg=0xfffff800032c47c0, frame=0xfffffe022815dac0) at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/kern/kern_fork.c:1011
>>>  #16 0xffffffff80d14f2e in fork_trampoline () at
>>>  /usr/home/adrian/work/freebsd/head/src/sys/amd64/amd64/exception.S:609
>>>  #17 0x0000000000000000 in ?? ()
>>>  Current language: auto; currently minimal
>>>
>>>  (kgdb) print *(struct llentry *)c_arg
>>>  $2 = {lle_next = {le_next = 0x0, le_prev = 0xfffff8005e867dc8},
>>>  r_l3addr = {addr4 = {s_addr = 16782508}, addr6 = {__u6_addr =
>>>  {__u6_addr8 = 0xfffff8005ecc4010 "�\024", __u6_addr16 =
>>>  0xfffff8005ecc4010,
>>>          __u6_addr32 = 0xfffff8005ecc4010}}}, ll_addr = {mac_aligned =
>>>  110869256150596, mac16 = 0xfffff8005ecc4020, mac8 = 0xfffff8005ecc4020
>>>  "D\036���d"}, spare0 = 0, spare1 = 0, lle_tbl = 0xfffff8005e867e00,
>>>    lle_head = 0xfffff8005e867dc8, lle_free = 0xffffffff80a2c5d0
>>>  <in_lltable_destroy_lle>, la_hold = 0x0, la_numheld = 0, la_expire =
>>>  2110, la_flags = 1, la_asked = 0, la_preempt = 5, ln_state = 0,
>>>  ln_router = 0, ln_ntick = 0,
>>>    lle_refcnt = 1, lle_chain = {le_next = 0x0, le_prev = 0x0},
>>>  lle_timer = {c_links = {le = {le_next = 0x0, le_prev =
>>>  0xffffffff81b2d588}, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0,
>>>  tqe_prev = 0xffffffff81b2d588}},
>>>      c_time = 9066299815445, c_precision = 322122525000, c_arg =
>>>  0xfffff8005ecc4000, c_func = 0xffffffff80a246e0 <arptimer>, c_lock =
>>>  0x0, c_flags = 0, c_iflags = 144, c_cpu = 0}, lle_lock = {lock_object
>>>  = {
>>>        lo_name = 0xffffffff8120fbce "lle", lo_flags = 90374144, lo_data
>>>  = 0, lo_witness = 0xfffffe0000b53c80}, rw_lock = 1}}
>>>
>>>  ..
>>>
>>>  (kgdb) print *((struct llentry *)c_arg)->lle_tbl
>>>  $4 = {llt_link = {sle_next = 0xdeadc0dedeadc0de}, llt_af = -559038242,
>>>  llt_hsize = -559038242, lle_head = 0xdeadc0dedeadc0de, llt_ifp =
>>>  0xdeadc0dedeadc0de, llt_lookup = 0xdeadc0dedeadc0de,
>>>    llt_alloc_entry = 0xdeadc0dedeadc0de, llt_delete_entry =
>>>  0xdeadc0dedeadc0de, llt_prefix_free = 0xdeadc0dedeadc0de,
>>>  llt_dump_entry = 0xdeadc0dedeadc0de, llt_hash = 0xdeadc0dedeadc0de,
>>>  llt_match_prefix = 0xdeadc0dedeadc0de,
>>>    llt_free_entry = 0xdeadc0dedeadc0de, llt_foreach_entry =
>>>  0xdeadc0dedeadc0de, llt_link_entry = 0xdeadc0dedeadc0de,
>>>  llt_unlink_entry = 0xdeadc0dedeadc0de, llt_fill_sa_entry =
>>>  0xdeadc0dedeadc0de,
>>>    llt_free_tbl = 0xdeadc0dedeadc0de}
>>>
>>>  :(
>>>
>>>  Any ideas on where next to look?
>>>
>>>  -adrian
>>>  _______________________________________________
>>>  freebsd-net_at_freebsd.org mailing list
>>>  https://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>  To unsubscribe, send any mail to "freebsd-net-unsubscribe_at_freebsd.org"
Received on Sat Oct 31 2015 - 14:01:22 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:00 UTC