panic with tcp timers

From: Gleb Smirnoff <glebius_at_FreeBSD.org>
Date: Thu, 16 Jun 2016 21:53:19 -0700
  Hi!

  At Netflix we are observing a race in TCP timers with head.
The problem is a regression, that doesn't happen on stable/10.
The panic usually happens after several hours at 55 Gbit/s of
traffic.

What happens is that tcp_timer_keep finds t_tcpcb being
NULL. Some coredumps have tcpcb already initialized,
with non-NULL t_tcpcb and in TCPS_ESTABLISHED state. Which
means that other CPU was working on the tcpcb while
the faulted one was working on the panic. So, this all looks
like a use after free, which conflicts with new allocation.

Comparing stable/10 and head, I see two changes that could
affect that:

- callout_async_drain
- switch to READ lock for inp info in tcp timers

That's why you are in To, Julien and Hans :)

We continue investigating, and I will keep you updated.
However, any help is welcome. I can share cores.

-- 
Totus tuus, Glebius.
Received on Fri Jun 17 2016 - 02:53:27 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:06 UTC