Re: 4.7 vs 5.2.1 SMP/UP bridging performance

Robert Watson wrote:
> Something I'd like to explore in detail is a careful comparison of
> performing delivery to completion in the interrupt thread vs using a set
> of netisr threads pinned to CPUs.  Avoiding lots of context switches is a
> good thing, but I don't have any measurements that suggest at what layer
> in the inbound packet path we get the most coallescing: we know interrupt
> mitigation and coallescing works well at a high level, but are we
> currently seeing substantial coallescing due to long-running interrupt
> threads, handoffs to netisrs, etc?  Some decent measurements with counters
> at each of the hand-off points might go a long way.

For the IP packet forwarding case you can profile normal forwarding against
ip_fastforwarding (netinet/ip_fastfwd.c).  IP fast forwarding processes to
completion from the interrupt thread.  If you don't have ipfw or pfil compiled
in or active the only locking it will do is for the routing table lookup.  To
get that down best is to have just one route, the default route.

sysctl -w net.inet.ip.fast_forwarding=1

Fast forwarding gives some nice speedups and reduces the variance in packet
processing quite a bit.  I've measured a reduction of CPU usage for forwarding
of approx. 40% on my old (now decomissioned) Athlon 900MHz with two fxp interfaces
with link0 enabled.  Haven't profiled on my new Dual-Opteron 1.4GHz with two bge
interfaces yet.

-- 
Andre