Re: TCP RST+data!

On Mon, 26 Nov 2007, Ian FREISLICH wrote:

> net.inet.tcp.delayed_ack=0
> net.inet.tcp.rfc1323=1
> http://www.freislich.nom.za/phone.nodelayedack
>
> net.inet.tcp.delayed_ack=1
> net.inet.tcp.rfc1323=1
> http://www.freislich.nom.za/phone.delayedack

Wow.  Wow!  WOW!

The TCP stack on that device amazes me in how poorly it is written.

Two things stand out even on the successful connection.

Note the lack of the use of TCP's sliding window - the phone sends a 
single packet, then waits for the ackknowledgement before sending another:

23:04:08.099170 IP 10.0.0.243.80 > 10.0.0.6.50287: P 1:10(9) ack 406 win 8192 <nop,nop,timestamp 5010 2471633>
23:04:08.099197 IP 10.0.0.6.50287 > 10.0.0.243.80: . ack 10 win 8324 <nop,nop,timestamp 2471924 5010>
23:04:08.102834 IP 10.0.0.243.80 > 10.0.0.6.50287: P 10:92(82) ack 406 win 8192 <nop,nop,timestamp 5010 2471924>
23:04:08.102843 IP 10.0.0.6.50287 > 10.0.0.243.80: . ack 92 win 8315 <nop,nop,timestamp 2471928 5010>
23:04:08.105059 IP 10.0.0.243.80 > 10.0.0.6.50287: P 92:112(20) ack 406 win 8192 <nop,nop,timestamp 5010 2471928>
23:04:08.105068 IP 10.0.0.6.50287 > 10.0.0.243.80: . ack 112 win 8323 <nop,nop,timestamp 2471930 5010>
23:04:08.107398 IP 10.0.0.243.80 > 10.0.0.6.50287: P 112:149(37) ack 406 win 8192 <nop,nop,timestamp 5010 2471930>

And when it's done, it closes the connection by sending a RST, instead of 
doing the normal TCP shutdown sequence with FINs:

23:04:08.125638 IP 10.0.0.243.80 > 10.0.0.6.50287: R 608:608(0) ack 406 win 8192

Comparing that vs the connection with delayed acks, the only difference I 
see is this:

Delayed acks:
23:03:01.852728 IP 10.0.0.6.57408 > 10.0.0.243.80: . ack 1 win 8326 <nop,nop,timestamp 2405682 4878>
23:03:01.852826 IP 10.0.0.6.57408 > 10.0.0.243.80: P 1:406(405) ack 1 win 8326 <nop,nop,timestamp 2405682 4878>
23:03:01.969360 IP 10.0.0.243.80 > 10.0.0.6.57408: . ack 406 win 7970 <nop,nop,timestamp 4878 2405682>
23:03:02.150649 IP 10.0.0.243.80 > 10.0.0.6.57408: P 1:10(9) ack 406 win 8192 <nop,nop,timestamp 4878 2405682>
23:03:02.156785 IP 10.0.0.243.80 > 10.0.0.6.57408: RP 10:608(598) ack 406 win 8192

No delayed acks:

23:04:07.808246 IP 10.0.0.6.50287 > 10.0.0.243.80: . ack 1 win 8326 <nop,nop,timestamp 2471633 5010>
23:04:07.808342 IP 10.0.0.6.50287 > 10.0.0.243.80: P 1:406(405) ack 1 win 8326 <nop,nop,timestamp 2471633 5010>
23:04:07.972124 IP 10.0.0.243.80 > 10.0.0.6.50287: . ack 406 win 8050 <nop,nop,timestamp 5010 2471633>
23:04:08.099170 IP 10.0.0.243.80 > 10.0.0.6.50287: P 1:10(9) ack 406 win 8192 <nop,nop,timestamp 5010 2471633>
23:04:08.099197 IP 10.0.0.6.50287 > 10.0.0.243.80: . ack 10 win 8324 <nop,nop,timestamp 2471924 5010>
23:04:08.102834 IP 10.0.0.243.80 > 10.0.0.6.50287: P 10:92(82) ack 406 win 8192 <nop,nop,timestamp 5010 2471924>

So it looks like if the phone does not receive an ACK for its data within 
.156785-.150649 = .006136 seconds, it gives up on the connection and 
closes it!  Yet it still manages to send data along with the RST, which is 
something that no other TCP stack (in common use) would do.

My guess is that even coming in from other OSes, you will experience 
problems if you go over a link with any noticeable latency.

Disabling delayed acks when you need to manage this phone looks like the 
only option; I don't see anything else that could be changed to better 
accomodate this phone.  I really hope that the TCP used by this phone is 
not used in any other devices.

-Mike