Re: Linux NFS ate my bge

From: Matthew Dillon <dillon_at_apollo.backplane.com>
Date: Wed, 22 Jul 2009 16:40:03 -0700 (PDT)
    TCP will likely work better, for several reasons, not the least of
    which being that the NFS client does not have to estimate a retransmit
    timeout on a rpc-by-rpc basis.  Such estimations fail utterly in the
    face of a large number of concurrent RPCs because latency winds up being
    governed by the disk backlog on the server.  A UDP mount will wind up
    retransmitting even under completely lossless conditions.

    Another reason TCP tends to work better is that UDP uses IP fragmentation
    and IP fragmentation reassembly is not typically in the critical path.
    The desired NFS filesystem block size is 16K (smaller will typically
    reduce performance), so even a 9000 MTU won't help.

    Also use netstat ... not sure what option, I think -x, to determine the
    actual size of the socket buffer being employed for the connection
    (TCP or UDP).  There are multiple internal caps in the kernel and it
    is often not as big as you might have thought it should be.  You want
    a 256KB socket buffer at a minimum for a GigE network.  Smaller works
    (at least for linear transfers), but you lose a lot of RPC concurrency
    from the client.  Again, something that matters more for a linux client
    vs a FreeBSD client.

						-Matt
Received on Wed Jul 22 2009 - 21:40:07 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:52 UTC