CURRENT r296381 somewere near tcp_detach?

From: Vitalij Satanivskij <satan_at_ukr.net>
Date: Sat, 5 Mar 2016 11:34:53 +0200
Hello.

Just after report about panic somewere in sendfile (http://docs.freebsd.org/cgi/getmsg.cgi?fetch=883140+0+current/freebsd-current),

and disabling sendfile functionality in software (nginx) I got another kernel panic (at last twice for this moment)

System message after reboot:

Mar  5 05:49:11 srv11 savecore: reboot after panic: tcp_detach: INP_TIMEWAIT && INP_DROPPED && tp != NULL
Mar  5 05:49:11 srv11 savecore: writing core to /var/crash/vmcore.2

kgdb kernel.debug /var/crash/vmcore.2 is :

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
panic: tcp_detach: INP_TIMEWAIT && INP_DROPPED && tp != NULL
cpuid = 11
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe1f9d1f4730
vpanic() at vpanic+0x182/frame 0xfffffe1f9d1f47b0
kassert_panic() at kassert_panic+0x126/frame 0xfffffe1f9d1f4820
tcp_usr_detach() at tcp_usr_detach+0x1bc/frame 0xfffffe1f9d1f4850
sofree() at sofree+0x1a6/frame 0xfffffe1f9d1f4880
tcp_close() at tcp_close+0x11e/frame 0xfffffe1f9d1f48b0
tcp_timer_2msl() at tcp_timer_2msl+0x278/frame 0xfffffe1f9d1f48e0
softclock_call_cc() at softclock_call_cc+0x1af/frame 0xfffffe1f9d1f49c0
softclock() at softclock+0x47/frame 0xfffffe1f9d1f49e0
intr_event_execute_handlers() at intr_event_execute_handlers+0x96/frame 0xfffffe1f9d1f4a20
ithread_loop() at ithread_loop+0xa6/frame 0xfffffe1f9d1f4a70
fork_exit() at fork_exit+0x84/frame 0xfffffe1f9d1f4ab0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe1f9d1f4ab0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
KDB: enter: panic

Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done.
done.
Loaded symbols for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/carp.ko...Reading symbols from /usr/lib/debug//boot/kernel/carp.ko.debug...done.
done.
Loaded symbols for /boot/kernel/carp.ko
Reading symbols from /boot/kernel/ums.ko...Reading symbols from /usr/lib/debug//boot/kernel/ums.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ums.ko
Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/tmpfs.ko
#0  doadump (textdump=0) at pcpu.h:221
221             __asm("movq %%gs:%1,%0" : "=r" (td)
(kgdb) bt
#0  doadump (textdump=0) at pcpu.h:221
#1  0xffffffff80384a0b in db_dump (dummy=<value optimized out>, dummy2=false, dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533
#2  0xffffffff803847fe in db_command (cmd_table=0x0) at /usr/src/sys/ddb/db_command.c:440
#3  0xffffffff80384594 in db_command_loop () at /usr/src/sys/ddb/db_command.c:493
#4  0xffffffff8038702b in db_trap (type=<value optimized out>, code=0) at /usr/src/sys/ddb/db_main.c:251
#5  0xffffffff80a656e3 in kdb_trap (type=3, code=0, tf=<value optimized out>) at /usr/src/sys/kern/subr_kdb.c:654
#6  0xffffffff80ea1298 in trap (frame=0xfffffe1f9d1f4660) at /usr/src/sys/amd64/amd64/trap.c:556
#7  0xffffffff80e81a77 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:234
#8  0xffffffff80a64dcb in kdb_enter (why=0xffffffff813b6c2f "panic", msg=0x80 <Address 0x80 out of bounds>) at cpufunc.h:63
#9  0xffffffff80a27b5f in vpanic (fmt=<value optimized out>, ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:750
#10 0xffffffff80a279b6 in kassert_panic (fmt=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:647
#11 0xffffffff80bf9bbc in tcp_usr_detach (so=<value optimized out>) at /usr/src/sys/netinet/tcp_usrreq.c:213
#12 0xffffffff80aad0b6 in sofree (so=0xfffff81820f89000) at /usr/src/sys/kern/uipc_socket.c:820
#13 0xffffffff80bf179e in tcp_close (tp=<value optimized out>) at /usr/src/sys/netinet/tcp_subr.c:1496
#14 0xffffffff80bf72f8 in tcp_timer_2msl (xtp=0xfffff81650263820) at /usr/src/sys/netinet/tcp_timer.c:374
#15 0xffffffff80a3d72f in softclock_call_cc (c=0xfffff81650263b68, cc=0xffffffff81d2db80, direct=0) at /usr/src/sys/kern/kern_timeout.c:723
#16 0xffffffff80a3dae7 in softclock (arg=<value optimized out>) at /usr/src/sys/kern/kern_timeout.c:861
#17 0xffffffff809ee7b6 in intr_event_execute_handlers (p=<value optimized out>, ie=0xfffff80114558d00) at /usr/src/sys/kern/kern_intr.c:1262
#18 0xffffffff809eee46 in ithread_loop (arg=0xfffff8011452fac0) at /usr/src/sys/kern/kern_intr.c:1275
#19 0xffffffff809ec074 in fork_exit (callout=0xffffffff809eeda0 <ithread_loop>, arg=0xfffff8011452fac0, frame=0xfffffe1f9d1f4ac0)
    at /usr/src/sys/kern/kern_fork.c:1034
#20 0xffffffff80e81fae in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:609
#21 0x0000000000000000 in ?? ()
Current language:  auto; currently minimal

kernel.debug and cores can be found here . http://hell.ukr.net/free/  (vmcore.1 and vmcore.2 for this panic)

Some addition info  If_igb, carp and pf used on server.


I would appreciate any kind of help in solving the problem.
Received on Sat Mar 05 2016 - 08:34:57 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:03 UTC