Re: Linux NFS ate my bge

From: Matthew Dillon <dillon_at_apollo.backplane.com>
Date: Wed, 22 Jul 2009 16:07:34 -0700 (PDT)
:requests.
:
:I've (re)discovered this now that I've recently got my assorted config 
:problems largely ironed out and have my FreeBSD box up to date again, I'm 
:reminded of an unresolved problem from way back, which is that my bge card 
:collapses after being subjected to a large amount of NFS traffic coming 
:from my Linux box, e.g. recompiling KDE on a discless workstation, which 
:has been responsible for three embuggerances so far today.

    If you are using a NFS UDP mount, try using a NFS TCP mount instead.
    This could very well instantly fix your issues even if it does not
    solve the underlying bugs.

    With a UDP mount the server can get a barrage of IP fragmented UDP
    packets, particularly from linux clients.  While I don't know any
    specific with regards to bugs in the bge driver itself, I do know that
    for a UDP mount to operate adequately over a GigE network the NFS server
    needs about a 2 Megabyte socket buffer to receive the bursts.

    This is not something that would typically be seen between two FreeBSD
    boxes as FreeBSD's asynchronous NFS client traffic (mainly read-ahead)
    is limited by available synchronous nfsiod threads, and further limited
    by default mount options, internal #defined concurrency limits, and
    by the out-of-order transmission of the read ahead rpcs caused by the
    way the nfsiod threads operate (which results in out-of-order processing
    on the server side and stalls the linear reads done by the client).

    Linux clients, on the other-hand can generate an enormous number of
    concurrent RPCs.

    Use of a TCP mount instead of a UDP mount solves the sockbuf and
    IP fragmentation issues.  The TCP connection will not use fragmented IP
    packets, will not blow away the server's receive-side sockbuf, and
    does a much better job dealing with any packet loss, to boot.

						-Matt
Received on Wed Jul 22 2009 - 21:07:38 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:52 UTC