Re: FYI: merging TCP, UDP, netisr locking changes

From: Robert Watson <rwatson_at_FreeBSD.org>
Date: Mon, 30 May 2011 10:53:22 +0100 (BST)
On Tue, 24 May 2011, Robert Watson wrote:

> Over the next few days, I will be merging a number of TCP-related locking 
> changes, as well as changes to various network stack infrastructure bits, 
> such as the netisr implementation.  The goal, generally, has been to move us 
> in the direction of supporting more clear CPU affinity for network flows, 
> the ability to program filters in network cards to support those affinities 
> explicitly, and elimination of cache line contention (whether by locks, 
> stats, etc) during high-volume parallel steady-state TCP load, with 
> ancillary benefits (hopefully) for UDP and other protocols.  This has 
> implied non-trivial changes to our inpcb locking model, netisr code, etc. 
> Detailed information will appear in commit messages as I go; some elements, 
> such a programming of card filters based on setting TCP socket options, are 
> very much a work in progress.
>
> Obviously, there are no bugs in this code at all.  However, if they are, 
> they might manifest as network problems, new WITNESS warnings, etc, and 
> network stack exercise + reports would be greatly appreciated!
>
> This work has been sponsored by Juniper Networks.  Thanks also to Bjoern 
> Zeeb, who has been reviewing changes!

After a series of smaller commits, I've just merged some initial decomposition 
of the pcbinfo lock into an additional pcbhash lock, which changes lock 
ordering and lookup with respect to inpcbs significantly (r222488; commit 
message below).  I expect there to be some initial instability as people shake 
out edge cases I didn't bump into in my testing.  Please report bugs to 
current_at_, and I'll pick them up there!

Robert N. M. Watson
University of Cambridge
Computer Laboratory



Decompose the current single inpcbinfo lock into two locks:

- The existing ipi_lock continues to protect the global inpcb list and
   inpcb counter.  This lock is now relegated to a small number of
   allocation and free operations, and occasional operations that walk
   all connections (including, awkwardly, certain UDP multicast receive
   operations -- something to revisit).

- A new ipi_hash_lock protects the two inpcbinfo hash tables for
   looking up connections and bound sockets, manipulated using new
   INP_HASH_*() macros.  This lock, combined with inpcb locks, protects
   the 4-tuple address space.

Unlike the current ipi_lock, ipi_hash_lock follows the individual inpcb
connection locks, so may be acquired while manipulating a connection on
which a lock is already held, avoiding the need to acquire the inpcbinfo
lock preemptively when a binding change might later be required.  As a
result, however, lookup operations necessarily go through a reference
acquire while holding the lookup lock, later acquiring an inpcb lock --
if required.

A new function in_pcblookup() looks up connections, and accepts flags
indicating how to return the inpcb.  Due to lock order changes, callers
no longer need acquire locks before performing a lookup: the lookup
routine will acquire the ipi_hash_lock as needed.  In the future, it will
also be able to use alternative lookup and locking strategies
transparently to callers, such as pcbgroup lookup.  New lookup flags are,
supplementing the existing INPLOOKUP_WILDCARD flag:

   INPLOOKUP_RLOCKPCB - Acquire a read lock on the returned inpcb
Decompose the current single inpcbinfo lock into two locks:

- The existing ipi_lock continues to protect the global inpcb list and
   inpcb counter.  This lock is now relegated to a small number of
   allocation and free operations, and occasional operations that walk
   all connections (including, awkwardly, certain UDP multicast receive
   operations -- something to revisit).

- A new ipi_hash_lock protects the two inpcbinfo hash tables for
   looking up connections and bound sockets, manipulated using new
   INP_HASH_*() macros.  This lock, combined with inpcb locks, protects
   the 4-tuple address space.

Unlike the current ipi_lock, ipi_hash_lock follows the individual inpcb
connection locks, so may be acquired while manipulating a connection on
which a lock is already held, avoiding the need to acquire the inpcbinfo
lock preemptively when a binding change might later be required.  As a
result, however, lookup operations necessarily go through a reference
acquire while holding the lookup lock, later acquiring an inpcb lock --
if required.

A new function in_pcblookup() looks up connections, and accepts flags
indicating how to return the inpcb.  Due to lock order changes, callers
no longer need acquire locks before performing a lookup: the lookup
routine will acquire the ipi_hash_lock as needed.  In the future, it will
also be able to use alternative lookup and locking strategies
transparently to callers, such as pcbgroup lookup.  New lookup flags are,
supplementing the existing INPLOOKUP_WILDCARD flag:

   INPLOOKUP_RLOCKPCB - Acquire a read lock on the returned inpcb
   INPLOOKUP_WLOCKPCB - Acquire a write lock on the returned inpcb

Callers must pass exactly one of these flags (for the time being).

Some notes:

- All protocols are updated to work within the new regime; especially,
   TCP, UDPv4, and UDPv6.  pcbinfo ipi_lock acquisitions are largely
   eliminated, and global hash lock hold times are dramatically reduced
   compared to previous locking.
- The TCP syncache still relies on the pcbinfo lock, something that we
   may want to revisit.
- Support for reverting to the FreeBSD 7.x locking strategy in TCP input
   is no longer available -- hash lookup locks are now held only very
   briefly during inpcb lookup, rather than for potentially extended
   periods.  However, the pcbinfo ipi_lock will still be acquired if a
   connection state might change such that a connection is added or
   removed.
- Raw IP sockets continue to use the pcbinfo ipi_lock for protection,
   due to maintaining their own hash tables.
- The interface in6_pcblookup_hash_locked() is maintained, which allows
   callers to acquire hash locks and perform one or more lookups atomically
   with 4-tuple allocation: this is required only for TCPv6, as there is no
   in6_pcbconnect_setup(), which there should be.
- UDPv6 locking remains significantly more conservative than UDPv4
   locking, which relates to source address selection.  This needs
   attention, as it likely significantly reduces parallelism in this code
   for multithreaded socket use (such as in BIND).
- In the UDPv4 and UDPv6 multicast cases, we need to revisit locking
   somewhat, as they relied on ipi_lock to stablise 4-tuple matches, which
   is no longer sufficient.  A second check once the inpcb lock is held
   should do the trick, keeping the general case from requiring the inpcb
   lock for every inpcb visited.
- This work reminds us that we need to revisit locking of the v4/v6 flags,
   which may be accessed lock-free both before and after this change.
- Right now, a single lock name is used for the pcbhash lock -- this is
   undesirable, and probably another argument is required to take care of
   this (or a char array name field in the pcbinfo?).

This is not an MFC candidate for 8.x due to its impact on lookup and
locking semantics.  It's possible some of these issues could be worked
around with compatibility wrappers, if necessary.

Reviewed by:    bz
Sponsored by:   Juniper Networks, Inc.
Received on Mon May 30 2011 - 07:53:23 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:14 UTC