Re: bizarre em + TSO + MSS issue in RELENG_7

On Nov 16, 2007 9:42 PM, Mike Andrews <mandrews_at_bit0.com> wrote:
> Has anyone run into problems with MSS not being respected when using TSO,
> specifically on em cards?
>
> I've run into a weird 3-way interaction with TSO, users with small MTU's
> (like 576), and a Cisco firewall that by default drops packets larger than
> the MSS for that session.  Most users can connect to our web servers
> behind this firewall just fine, but a very few couldn't since we moved
> from RELENG_6 to RELENG_7 about two weeks ago.  One of them could connect
> again after removing a Linksys box at his end that was clamping the MSS at
> 536.  Some of the others were on dialup which tends to use 576 MTU / 536
> MSS.  The firewall was dropping a lot of *outbound* traffic to these users
> because the MSS was exceeded -- ~1000 byte packets for a 536 MSS
> connection.  All of those users' problems disappeared after disabling TSO
> on our end.  Pretty weird. :)
>
> I'm quite happy running without TSO and I really don't expect this to get
> looked at before 7.0-RELEASE -- I just thought the info that MSS appears
> (to me anyway) to not always be respected might be useful for those
> looking at TSO-related throughput issues.  In our case throughput isn't
> the issue, it's certain connections stalling out -- it actually behaves
> sort of like (but isn't) a PMTUD issue.
>
> I've got a lot more detail for whoever wants it (Jack Vogel, probably)
> including tcpdumps and firewall logs but I thought I'd put it out here
> first in case it wasn't em-specific rather than clutter the list right
> away, or if it's a known issue that's already fixed in HEAD or something.
> This is with em driver version 6.5.3, 7.0-BETA2 from Nov 2.


Hrrm. Either TCP isn't setting tso_segsz correctly, or em isn't
honoring tso_segsz. Can you add some instrumentation to the TSO code
in em to see if tso_segsz is being set to the MSS?

 -Kip