Re: tcp hostcache and ip fastforward for review

On Nov 9, 2003, at 2:47 PM, Andre Oppermann wrote:

> Jonathan Mini wrote:
>>
>> On Nov 9, 2003, at 8:19 AM, Andre Oppermann wrote:
>>
>>>   - DoS attack 2: make MSS very low on local side of connection
>>>     and send maaaany small packet to remote host. For every packet
>>>     (eg. 2 bytes payload) a sowakeup is done to the listening
>>>     process. Consumes a lot of CPU there.
>>>
>>
>> This sounds as if it might be worthwhile to add a delay to
>> the TF_NODELAY case for receive processing as well.
>
> Unfortunatly it is not that easy. We can't just do that unconditionally
> to all connections. It would probably break or delay many things. You
> never know how much data is outstanding and whether it's just this
> packet with 2 bytes outstanding...

This would be disastrous to the performance of interactive
sockets, however theoretically those connections have
NODELAY set. My above comment is a bit confusing: I meant the
"non TF_NODELAY" case, that is when Nagling is enabled.

In this situation, you would be delay an sowakeup until
either a timeout or SO_RCVLOWAT-set value was reached.  The normal
SO_RCVLOWAT case delays until SO_RCVTIMEO is reached.  I suppose
the application could simulate this with a large SO_RCVLOWAT and a
small SO_RCVTIMEO, but I was wondering about the effects of such a
change as part of !TF_NODELAY.

Sadly, there's this PSH bit in the TCP header that's completely
unreliable and could be used for scenarios like this.

> As an application aware of this problematic you have currently two
> options: use accept filters (FreeBSD only) or set SO_RCVLOWAT to some
> higher value than the default 1 byte. Only the first one is workable
> if you don't know what and how much the clients send to you. Relying
> on the application to activate any such option to prevent this kind
> of DoS is unfortunatly whishful thinking.

I was not suggesting that we use this to counter an attack, only asking
if it might be a worthwhile performance optimization to consider.
This is an unlikely case (many small packets sent to a non-interactive
application), so I can't see the improvement as being globally useful.

> The code I've put in here simply caps off the extreme cases. It
> counts all packets and bytes in any given second and computes the
> average payload size per packet. If that is less than we have defined
> for minmss it will reset and drop the connection. However it will only
> start to compute the average if there are more than 1'000 packets per
> second on the same tcp connection. I've chosen this quite high value
> to never disconnect any ligitimate connection which just happens to
> send many small packets. In my tests I've seen telnet/ssh sending
> close to 100 small packets per second (some large copy-pasting and
> cat'ing of many small files). Probably 500 packets per second is a
> better cut-off value but I just want to be sure to never hit a false
> positive.

This is actually a small value for TCP connections which are being
used to forward messages, especially on gigabit links.  
Heavily-intensive
web applications that are using small HTTP requests (pipelined inside a
persistent connection) to update small manipulations of state are
a good example of this.  I wouldn't be surprised to see chatter
between SQL servers follow similar patterns.  Applications which
use XML-based messaging often send several small packets per message,
which is unfortunate.

On the other hand, I'm used to looking at proxies, which are not
the general case.  This is why the limits are tunable, after all. =)

-- 
Jonathan Mini
mini_at_freebsd.org
http://www.freebsd.org