Re: Some performance measurements on the FreeBSD network stack

On 20.04.2012 10:26, Alexander V. Chernikov wrote:
> On 20.04.2012 01:12, Andre Oppermann wrote:
>> On 19.04.2012 22:34, K. Macy wrote:
>>> If the number of peers is bounded then you can use the flowtable. Max
>>> PPS is much higher bypassing routing lookup. However, it doesn't scale
 >
>  From my experience, turning fastfwd on gives ~20-30% performance
> increase (10G forwarding with firewalling, 1.4MPPS). ip_forward() uses 2
> lookups (ip_rtaddr + ip_output) vs 1 ip_fastfwd().

Another difference is the packet copy the normal forwarding path
does to be able to send a ICMP redirect message if the packet is
forwarded to a different gateway on the same LAN.  fastforward
doesn't do that.

> The worst current problem IMHO is number of locks packet have to
> traverse, not number of lookups.

Agreed.  Actually the locking in itself is not the problem.  It's
the side effects of cache line dirtying/bouncing and contention.
However in the great majority of the cases the data protected by
the lock is only read, not modified making a 'full' lock expensive.

-- 
Andre