Re: NFSv3 issues with latest -current

On Sun, 29 Oct 2017 18:11:41 +0300, Yuri Pankov wrote:
> On Sun, 29 Oct 2017 13:13:31 +0000, Rick Macklem wrote:
>> Yuri Pankov wrote:
>>> All file operations (e.g. copying the file over NFSv3 for me) seem to be
>>> stuck running the latest -current (r325100).  Reverting just the kernel
>>> to r323779 (arbitrary chosen) seems to help.  I noticed the "Stale file
>>> handle when mounting nfs" message but I don't get the "stale file
>>> handle" messages from mount, probably as I'm not running any linux clients.
>> These kinds of problems are usually related to your net interface device
>> driver or the TCP stack.
>>
>> A couple of things to try:
>> - Disable TSO (look for a sysctl with "tso" in it).
>> - Try using mount options rsize=32768,waize=32768 to reduce the I/O
>>     size. Some device drivers don't handle long chains of mbufs well,
>>     especially when the size is near 64K.
>> (These issues have been fixed in current, but if a bug slips into a net driver
>>    update or ???)
>> - Look at recent changes to the net device driver you are using and try reverting
>>     those changes if you can do so.
>> - Capture packets and look at them in wireshark (which knows NFS) and see
>>     what is going on the wire.
>>
>> There hasn't been any recent changes to NFS that should affect NFSv3 mounts
>> or to the kernel rpc, so I doubt the NFSv4.1 changes would be involved.
> 
> Thanks for the hints, Rick!
> 
> Indeed, it was one of the changes to sys/dev/e1000, reverting just the
> commit made everything look normal again (CC'ing the author).

One thing I forgot to mention here, the problem is visible only if the 
client side has MTU of 1500 configured; when both sides have MTU 9000, 
everything looks to be normal -- noticed this when my XenServer (having 
MTU of 1500 on management interface) wasn't able to read the ISO from 
NFS datastore.

> The NIC is:
> 
> igb0_at_pci0:2:0:0:        class=0x020000 card=0x10c915d9 chip=0x10c98086
> rev=0x01 hdr=0x00
>       vendor     = 'Intel Corporation'
>       device     = '82576 Gigabit Network Connection'
>       class      = network
>       subclass   = ethernet
> 
> Interface configuration (note the MTU):
> 
> igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
> options=e525bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
>           ether 00:25:90:72:54:22
>           inet6 fe80::225:90ff:fe72:5422%igb0 prefixlen 64 scopeid 0x1
>           inet 192.168.1.4 netmask 0xffffff00 broadcast 192.168.1.255
>           nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
>           media: Ethernet autoselect (1000baseT <full-duplex>)
>           status: active
> 
> And the commit itself:
> 
> commit f81cb8df32ae96299b8bbc2e948c17ad3aab59ca
> Author: shurd <shurd_at_FreeBSD.org>
> Date:   Sat Sep 23 01:33:20 2017 +0000
> 
>       Some small packet performance improvements
> 
>       If the packet is smaller than MTU, disable the TSO flags.
>       Move TCP header parsing inside the IS_TSO?() test.
>       Add a new IFLIB_NEED_ZERO_CSUM flag to indicate the checksums need
> to be zeroed before TX.
> 
>       Reviewed by:    sbruno
>       Approved by:    sbruno (mentor)
>       Sponsored by:   Limelight Networks
>       Differential Revision:  https://reviews.freebsd.org/D12442
> 
> Notes:
>       svn path=/head/; revision=323941