Re: serious networking (em) performance (ggate and NFS) problem

On Thu, 18 Nov 2004, Pawel Jakub Dawidek wrote:

> On Wed, Nov 17, 2004 at 11:57:41PM +0100, Emanuel Strobl wrote:
> +> Dear best guys,
> +> 
> +> I really love 5.3 in many ways but here're some unbelievable transfer rates, 
> +> after I went out and bought a pair of Intel GigaBit Ethernet Cards to solve 
> +> my performance problem (*laugh*):
> [...]
> 
> I done some test in the past with ggate and PCI64/GBit NICs and I get
> ~38MB/s AFAIR. 
> 
> Remember that when using 32bit PCIs you can get transfer about
> 500Mbit/s. 
> 
> Please run those test with netperf (/usr/ports/benchmarks/netperf) and
> send results. 

Be aware, btw, that while netperf is a pretty decent tool, it performs a
lot of socket select and timing operations, so isn't always a good measure
of maximum capabilities of a system.  I.e., it is not unusual to see that
a netperf send test will only see send() as one in three or one in four
system calls -- as a result, it uses a measurable amount of CPU on things
other than sending.  In an environment with CPU constraints (slower CPU or
faster network), this can impact the performance results substantially.

For example, when measuring maximum packet send performance using minimal
packet sizes from user space, several of my test boxes are constrained
with 64-bit gig-e PCI cards based on CPU.  In particular, the combined
cost of the additional system calls and operations cuts into available CPU
for send.  By eliminating the misc.  overheads of netperf using netsend, I
can improve performance by 20%-30%. 

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert_at_fledge.watson.org      Principal Research Scientist, McAfee Research