Re: bizarre em + TSO + MSS issue in RELENG_7

Kip Macy wrote:
> On Nov 17, 2007 3:23 PM, Mike Andrews <mandrews_at_bit0.com> wrote:
>> On Sat, 17 Nov 2007, Kip Macy wrote:
>>
>>> On Nov 17, 2007 2:33 PM, Mike Andrews <mandrews_at_bit0.com> wrote:
>>>> On Sat, 17 Nov 2007, Kip Macy wrote:
>>>>
>>>>> On Nov 17, 2007 10:33 AM, Denis Shaposhnikov <dsh_at_vlink.ru> wrote:
>>>>>> On Sat, 17 Nov 2007 00:42:54 -0500 (EST)
>>>>>> Mike Andrews <mandrews_at_bit0.com> wrote:
>>>>>>
>>>>>>> Has anyone run into problems with MSS not being respected when using
>>>>>>> TSO, specifically on em cards?
>>>>>> Yes, I wrote about this problem on the beginning of 2007, see
>>>>>>
>>>>>>     http://tinyurl.com/3e5ak5
>>>>>>
>>>>> if_em.c:3502
>>>>>        /*
>>>>>         * Payload size per packet w/o any headers.
>>>>>         * Length of all headers up to payload.
>>>>>         */
>>>>>        TXD->tcp_seg_setup.fields.mss = htole16(mp->m_pkthdr.tso_segsz);
>>>>>        TXD->tcp_seg_setup.fields.hdr_len = hdr_len;
>>>>>
>>>>>
>>>>> Please print out the value of tso_segsz here. It appears to be being
>>>>> set correctly. The only thing I can think of is that t_maxopd is not
>>>>> correct. As tso_segsz is correct here:
>>>>
>>>> It repeatedly prints 1368 during a 1 meg file transfer over a connection
>>>> with a 1380 MSS.  Any other printf's I can add?  I'm working on a web page
>>>> with tcpdump / firewall log output illustrating the issue...
>>> Mike -
>>> Denis' tcpdump output doesn't show oversized segments, something else
>>> appears to be happening there. Can you post your tcpdump output
>>> somewhere?
>>
>> URL sent off-list.
> 
>        if (tso) {
>                m->m_pkthdr.csum_flags = CSUM_TSO;
>                m->m_pkthdr.tso_segsz = tp->t_maxopd - optlen;
>        }
> 
> 
> Please print the value of maxopd and optlen under "if (tso)" in
> tcp_output. I think the calculated optlen may be too small.


maxopt=1380 - optlen=12 = tso_segsz=1368

Weird though, after this reboot, I had to re-copy a 4 meg file 5 times 
to start getting the firewall to log any drops.  Transfer rate was 
around 240KB/sec before the firewall started to drop, then it went down 
to about 64KB/sec during the 5th copy, and stayed there for subsequent 
copies.  The actual packet size the firewall said it was dropping was 
varying all over the place still, yet the maxopt/optlen/tso_segsz values 
stayed constant.  But it's interesting that it didn't start dropping 
immediately after the reboot -- though the transfer rate was still 
sub-optimal.