Re: HELP! weird tcp problem

From: <youshi10_at_u.washington.edu>
Date: Mon, 18 Jun 2007 12:42:37 -0700 (PDT)
On Mon, 18 Jun 2007, Maksim Yevmenkin wrote:

> All,
>
> after upgrading to -current this weekend i started to have weird tcp
> problems. first of all tcp connections from initiated from firefox
> and/or thunderbird no longer work. please see info below
>
> ====
>
> beetle% uname -a
> FreeBSD beetle.level3.com 7.0-CURRENT FreeBSD 7.0-CURRENT #91: Sat Jun
> 16 13:33:48 PDT 2007
> root_at_beetle.level3.com:/usr/obj/usr/src/sys/GENERIC  i386
>
> ====
>
> beetle% sysctl net.inet.tcp
> net.inet.tcp.rfc1323: 1
> net.inet.tcp.mssdflt: 512
> net.inet.tcp.keepidle: 120000
> net.inet.tcp.keepintvl: 75000
> net.inet.tcp.sendspace: 32768
> net.inet.tcp.recvspace: 65536
> net.inet.tcp.keepinit: 75000
> net.inet.tcp.delacktime: 100
> net.inet.tcp.v6mssdflt: 1024
> net.inet.tcp.hostcache.purge: 0
> net.inet.tcp.hostcache.prune: 300
> net.inet.tcp.hostcache.expire: 3600
> net.inet.tcp.hostcache.count: 1
> net.inet.tcp.hostcache.bucketlimit: 30
> net.inet.tcp.hostcache.hashsize: 512
> net.inet.tcp.hostcache.cachelimit: 15360
> net.inet.tcp.recvbuf_max: 262144
> net.inet.tcp.recvbuf_inc: 16384
> net.inet.tcp.recvbuf_auto: 1
> net.inet.tcp.insecure_rst: 0
> net.inet.tcp.rfc3390: 1
> net.inet.tcp.rfc3042: 1
> net.inet.tcp.drop_synfin: 0
> net.inet.tcp.delayed_ack: 1
> net.inet.tcp.blackhole: 0
> net.inet.tcp.log_in_vain: 0
> net.inet.tcp.sendbuf_max: 262144
> net.inet.tcp.sendbuf_inc: 8192
> net.inet.tcp.sendbuf_auto: 1
> net.inet.tcp.tso: 1
> net.inet.tcp.newreno: 1
> net.inet.tcp.local_slowstart_flightsize: 4
> net.inet.tcp.slowstart_flightsize: 1
> net.inet.tcp.path_mtu_discovery: 1
> net.inet.tcp.reass.overflows: 0
> net.inet.tcp.reass.maxqlen: 48
> net.inet.tcp.reass.cursegments: 0
> net.inet.tcp.reass.maxsegments: 1600
> net.inet.tcp.sack.globalholes: 0
> net.inet.tcp.sack.globalmaxholes: 65536
> net.inet.tcp.sack.maxholes: 128
> net.inet.tcp.sack.enable: 1
> net.inet.tcp.inflight.stab: 20
> net.inet.tcp.inflight.max: 1073725440
> net.inet.tcp.inflight.min: 6144
> net.inet.tcp.inflight.rttthresh: 10
> net.inet.tcp.inflight.debug: 0
> net.inet.tcp.inflight.enable: 1
> net.inet.tcp.isn_reseed_interval: 0
> net.inet.tcp.icmp_may_rst: 1
> net.inet.tcp.pcbcount: 8
> net.inet.tcp.do_tcpdrain: 1
> net.inet.tcp.tcbhashsize: 512
> net.inet.tcp.minmss: 216
> net.inet.tcp.syncache.rst_on_sock_fail: 1
> net.inet.tcp.syncache.rexmtlimit: 3
> net.inet.tcp.syncache.hashsize: 512
> net.inet.tcp.syncache.count: 0
> net.inet.tcp.syncache.cachelimit: 15360
> net.inet.tcp.syncache.bucketlimit: 30
> net.inet.tcp.syncookies_only: 0
> net.inet.tcp.syncookies: 1
> net.inet.tcp.timer_race: 0
> net.inet.tcp.finwait2_timeout: 60000
> net.inet.tcp.fast_finwait2_recycle: 0
> net.inet.tcp.always_keepalive: 1
> net.inet.tcp.rexmit_slop: 200
> net.inet.tcp.rexmit_min: 3
> net.inet.tcp.msl: 30000
> net.inet.tcp.nolocaltimewait: 0
> net.inet.tcp.maxtcptw: 2465
>
> ====
>
> beetle% ifconfig -a
> bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>       options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
>       ether 00:11:43:54:22:c9
>       inet 10.12.163.152 netmask 0xffffff80 broadcast 10.12.163.255
>       media: Ethernet 100baseTX <full-duplex>
>       status: active
> plip0: flags=108810<POINTOPOINT,SIMPLEX,MULTICAST,NEEDSGIANT> metric 0 mtu 1500
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>       inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
>       inet6 ::1 prefixlen 128
>       inet 127.0.0.1 netmask 0xff000000
>
> ====
>
> beetle% netstat -rn
> Routing tables
>
> Internet:
> Destination        Gateway            Flags    Refs      Use  Netif Expire
> default            10.12.163.129      UGS         0      107   bge0
> 10.12.163.128/25   link#1             UC          0        0   bge0
> 10.12.163.129      00:1a:6c:74:2b:39  UHLW        2        0   bge0   1143
> 10.12.163.151      00:16:d4:9e:a5:b0  UHLW        1      150   bge0   1147
> 10.12.163.199      08:00:20:b5:8f:92  UHLW        2        2   bge0    861
> 127.0.0.1          127.0.0.1          UH          0       94    lo0
> 192.168.1.0/24     10.12.163.199      UGS         0        2   bge0
>
> Internet6:
> Destination                       Gateway                       Flags
>   Netif Expire
> ::1                               ::1                           UHL         lo0
> fe80::%lo0/64                     fe80::1%lo0                   U           lo0
> fe80::1%lo0                       link#3                        UHL         lo0
> ff01:3::/32                       fe80::1%lo0                   UC          lo0
> ff02::%lo0/32                     fe80::1%lo0                   UC          lo0
>
> ====
>
> beetle# ipfw show
> 00100  188  21618 allow ip from any to any via lo0
> 00200    0      0 deny ip from any to 127.0.0.0/8
> 00300    0      0 deny ip from 127.0.0.0/8 to any
> 65000 1103 163395 allow ip from any to any
> 65535    0      0 deny ip from any to any
>
> ===
>
> now tcpdump output when i try to go to http://www.yahoo.com/ from firefox
>
> beetle# tcpdump -n -vv -s0 -S -X -i bge0 port 80
> tcpdump: listening on bge0, link-type EN10MB (Ethernet), capture size
> 65535 bytes
> 10:34:47.217922 IP (tos 0x0, ttl  64, id 459, offset 0, flags [DF],
> proto: TCP (6), length: 60, bad cksum 0 (->5c59)!) 10.12.163.152.49879
>> 209.191.93.52.80: S, cksum 0xdcc6 (incorrect (-> 0x7442),
> 3892604697:3892604697(0) win 65535 <mss 1460,nop,wscale
> 8,sackOK,timestamp 384217 0>
>       0x0000:  4500 003c 01cb 4000 4006 0000 0a0c a398  E..<.._at_._at_.......
>       0x0010:  d1bf 5d34 c2d7 0050 e804 6f19 0000 0000  ..]4...P..o.....
>       0x0020:  a002 ffff dcc6 0000 0204 05b4 0103 0308  ................
>       0x0030:  0402 080a 0005 dcd9 0000 0000            ............
> 10:34:47.273178 IP (tos 0x0, ttl  51, id 60635, offset 0, flags [DF],
> proto: TCP (6), length: 64) 209.191.93.52.80 > 10.12.163.152.49879: S,
> cksum 0x2b67 (correct), 112085153:112085153(0) ack 3892604698 win
> 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 2476823771
> 384217,sackOK,eol>
>       0x0000:  4500 0040 ecdb 4000 3306 7e44 d1bf 5d34  E.._at_.._at_.3.~D..]4
>       0x0010:  0a0c a398 0050 c2d7 06ae 48a1 e804 6f1a  .....P....H...o.
>       0x0020:  b012 ffff 2b67 0000 0204 05b4 0103 0301  ....+g..........
>       0x0030:  0101 080a 93a1 54db 0005 dcd9 0402 0000  ......T.........
> 10:34:47.273219 IP (tos 0x0, ttl  64, id 460, offset 0, flags [DF],
> proto: TCP (6), length: 40, bad cksum 0 (->5c6c)!) 10.12.163.152.49879
>> 209.191.93.52.80: R, cksum 0xdcb2 (incorrect (-> 0xb901),
> 3892604698:3892604698(0) win 0
>       0x0000:  4500 0028 01cc 4000 4006 0000 0a0c a398  E..(.._at_._at_.......
>       0x0010:  d1bf 5d34 c2d7 0050 e804 6f1a 0000 0000  ..]4...P..o.....
>       0x0020:  5004 0000 dcb2 0000 0000 0000 0000       P.............
> ^C
> 3 packets captured
> 255 packets received by filter
> 0 packets dropped by kernel
>
> as you can see my laptop simply sends reset after syn+ack is recevied
> from the peer.
>
> ====
>
> now tcpdump output when i do "telnet www.yahoo.com 80". as you can see
> everything seems to work.
>
> beetle# tcpdump -n -vv -s0 -S -X -i bge0 port 80
> tcpdump: listening on bge0, link-type EN10MB (Ethernet), capture size
> 65535 bytes
> 10:35:15.462718 IP (tos 0x10, ttl  64, id 478, offset 0, flags [DF],
> proto: TCP (6), length: 60, bad cksum 0 (->5c36)!) 10.12.163.152.56333
>> 209.191.93.52.80: S, cksum 0xdcc6 (incorrect (-> 0x84e9),
> 1764906425:1764906425(0) win 65535 <mss 1460,nop,wscale
> 8,sackOK,timestamp 412462 0>
>       0x0000:  4510 003c 01de 4000 4006 0000 0a0c a398  E..<.._at_._at_.......
>       0x0010:  d1bf 5d34 dc0d 0050 6932 55b9 0000 0000  ..]4...Pi2U.....
>       0x0020:  a002 ffff dcc6 0000 0204 05b4 0103 0308  ................
>       0x0030:  0402 080a 0006 4b2e 0000 0000            ......K.....
> 10:35:15.517030 IP (tos 0x0, ttl  51, id 11180, offset 0, flags [DF],
> proto: TCP (6), length: 64) 209.191.93.52.80 > 10.12.163.152.56333: S,
> cksum 0x3539 (correct), 1090115527:1090115527(0) ack 1764906426 win
> 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 2478346791
> 412462,sackOK,eol>
>       0x0000:  4500 0040 2bac 4000 3306 3f74 d1bf 5d34  E.._at_+._at_.3.?t..]4
>       0x0010:  0a0c a398 0050 dc0d 40f9 d7c7 6932 55ba  .....P.._at_...i2U.
>       0x0020:  b012 ffff 3539 0000 0204 05b4 0103 0301  ....59..........
>       0x0030:  0101 080a 93b8 9227 0006 4b2e 0402 0000  .......'..K.....
> 10:35:15.517077 IP (tos 0x10, ttl  64, id 479, offset 0, flags [DF],
> proto: TCP (6), length: 52, bad cksum 0 (->5c3d)!) 10.12.163.152.56333
>> 209.191.93.52.80: ., cksum 0xdcbe (incorrect (-> 0x73ca),
> 1764906426:1764906426(0) ack 1090115528 win 260 <nop,nop,timestamp
> 412516 2478346791>
>       0x0000:  4510 0034 01df 4000 4006 0000 0a0c a398  E..4.._at_._at_.......
>       0x0010:  d1bf 5d34 dc0d 0050 6932 55ba 40f9 d7c8  ..]4...Pi2U._at_...
>       0x0020:  8010 0104 dcbe 0000 0101 080a 0006 4b64  ..............Kd
>       0x0030:  93b8 9227                                ...'
> 10:35:20.564276 IP (tos 0x10, ttl  64, id 482, offset 0, flags [DF],
> proto: TCP (6), length: 52, bad cksum 0 (->5c3a)!) 10.12.163.152.56333
>> 209.191.93.52.80: F, cksum 0xdcbe (incorrect (-> 0x6012),
> 1764906426:1764906426(0) ack 1090115528 win 260 <nop,nop,timestamp
> 417563 2478346791>
>       0x0000:  4510 0034 01e2 4000 4006 0000 0a0c a398  E..4.._at_._at_.......
>       0x0010:  d1bf 5d34 dc0d 0050 6932 55ba 40f9 d7c8  ..]4...Pi2U._at_...
>       0x0020:  8011 0104 dcbe 0000 0101 080a 0006 5f1b  .............._.
>       0x0030:  93b8 9227                                ...'
> 10:35:20.618435 IP (tos 0x0, ttl  51, id 18390, offset 0, flags [DF],
> proto: TCP (6), length: 52) 209.191.93.52.80 > 10.12.163.152.56333: .,
> cksum 0xcb10 (correct), 1090115528:1090115528(0) ack 1764906427 win
> 33304 <nop,nop,timestamp 2478351892 417563>
>       0x0000:  4500 0034 47d6 4000 3306 2356 d1bf 5d34  E..4G._at_.3.#V..]4
>       0x0010:  0a0c a398 0050 dc0d 40f9 d7c8 6932 55bb  .....P.._at_...i2U.
>       0x0020:  8010 8218 cb10 0000 0101 080a 93b8 a614  ................
>       0x0030:  0006 5f1b                                .._.
> 10:35:20.618588 IP (tos 0x0, ttl  51, id 18391, offset 0, flags [DF],
> proto: TCP (6), length: 52) 209.191.93.52.80 > 10.12.163.152.56333: F,
> cksum 0xcb0e (correct), 1090115528:1090115528(0) ack 1764906427 win
> 33304 <nop,nop,timestamp 2478351893 417563>
>       0x0000:  4500 0034 47d7 4000 3306 2355 d1bf 5d34  E..4G._at_.3.#U..]4
>       0x0010:  0a0c a398 0050 dc0d 40f9 d7c8 6932 55bb  .....P.._at_...i2U.
>       0x0020:  8011 8218 cb0e 0000 0101 080a 93b8 a615  ................
>       0x0030:  0006 5f1b                                .._.
> 10:35:20.618623 IP (tos 0x10, ttl  64, id 483, offset 0, flags [DF],
> proto: TCP (6), length: 52, bad cksum 0 (->5c39)!) 10.12.163.152.56333
>> 209.191.93.52.80: ., cksum 0xdcbe (incorrect (-> 0x4bed),
> 1764906427:1764906427(0) ack 1090115529 win 260 <nop,nop,timestamp
> 417617 2478351893>
>       0x0000:  4510 0034 01e3 4000 4006 0000 0a0c a398  E..4.._at_._at_.......
>       0x0010:  d1bf 5d34 dc0d 0050 6932 55bb 40f9 d7c9  ..]4...Pi2U._at_...
>       0x0020:  8010 0104 dcbe 0000 0101 080a 0006 5f51  .............._Q
>       0x0030:  93b8 a615                                ....
> ^C
> 7 packets captured
> 201 packets received by filter
> 0 packets dropped by kernel
>
> does anyone knows what is going on here?!
>
> thanks,
> max
>
> p.s. everything was working just fine prior to upgrade this weekend

I've noticed similar actually on my server. ICMP works just fine, and so does fetching from ports. Mozilla's stuff just appears to be hosed for some odd reason...

I have an sis card on my server, with no special net sysctls set except for the blackhole one I think.

I'll take a look and report back tonight.

-Garrett
Received on Mon Jun 18 2007 - 17:42:39 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:12 UTC