Automatic TCP send and receive socket buffer sizing

From: Andre Oppermann <andre_at_freebsd.org>
Date: Tue, 12 Dec 2006 23:30:26 +0100
This is a patch adding automatic TCP send and receive socket buffer sizing.
Normally the socket buffers are static (either derived from global defaults
or set with setsockopt) and do not adapt to real network conditions.  Two
things happen: a) your socket buffers are too small and you can't reach the
full potential of the network between both hosts; b) your socket buffers are
too big and you waste a lot of kernel memory for data just sitting around.

With automatic TCP send and receive socket buffers we can start with a small
buffer and quickly grow it in parallel with the TCP congestion window to match
real network conditions.

FreeBSD has a default 32K send socket buffer.  This supports a maximal
transfer rate of only slightly more than 2Mbit/s on a 100ms RTT trans-
continental link.  Or at 200ms just above 1Mbit/s.  With TCP send buffer
auto scaling and the default values below it supports 20Mbit/s at 100ms
and 10Mbit/s at 200ms.  That's an improvement of factor 10, or 1000%.
For the receive side it looks slightly better with a default of 64K buffer
size.

The automatic send buffer sizing patch is currently running on one half of
the FTP.FreeBSD.ORG cluster w/o any problems so far.  Against this machine
with the automatic receive buffer sizing patch I can download at 5.7MBytes
per second.  Without patch it maxed out at 1.6MBytes per second as the delay
bandwidth product became equal to the static socket buffer size without hitting
the limits of the physical link between the machines.  My test machine is about
35ms from that FTP.FreeBSD.ORG and connected through a moderately loaded 100Mbit
Internet link.

New sysctl's are:

  net.inet.tcp.sendbuf_auto=1	   (enabled)
  net.inet.tcp.sendbuf_inc=8192	   (8K, step size)
  net.inet.tcp.sendbuf_max=262144   (256K, growth limit)
  net.inet.tcp.recvbuf_auto=1	   (enabled)
  net.inet.tcp.recvbuf_inc=16384	   (16K, step size)
  net.inet.tcp.recvbuf_max=262144   (256K, growth limit)

The patch is available here (it may apply with some fuzz):

  http://people.freebsd.org/~andre/tcp_auto_buf-20061212.diff

Any tests and test reports are very welcome.

-- 
Andre
Received on Tue Dec 12 2006 - 22:03:39 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:03 UTC