Re: datapoints on 10G throughput with TCP ?

From: Luigi Rizzo <rizzo_at_iet.unipi.it>
Date: Tue, 6 Dec 2011 00:31:01 +0100
On Mon, Dec 05, 2011 at 03:08:54PM -0800, Jack Vogel wrote:
> You can't get line rate with ixgbe, in what configuration/hardware?
> We surely do get line rate in validation here, but its sensitive to
> your hardware and config.

sources from HEAD as of a week or so, default parameter setting,
82599 on an Intel dual port 10G card, Intel i7-870 CPU (4 cores)
at 2.93 GHz, on asus MB and the card on a PCIe-x16 slot, MTU=1500 bytes.
Same hardware, same defaults and nuttcp on linux does 8.5 Gbit/s.

I can do line rate with a single flow if i use MTU=9000 and set
max_interrupt_rate=0 (even reducing the CPU speed to 1.2 GHz).

I can saturate the link with multiple flows (say nuttcp -N 8).

cheers
luigi

> Jack
> 
> 
> On Mon, Dec 5, 2011 at 2:28 PM, Luigi Rizzo <rizzo_at_iet.unipi.it> wrote:
> 
> > On Mon, Dec 05, 2011 at 11:15:09PM +0200, Daniel Kalchev wrote:
> > >
> > > On Dec 5, 2011, at 9:27 PM, Luigi Rizzo wrote:
> > >
> > > > - have two machines connected by a 10G link
> > > > - on one run "nuttcp -S"
> > > > - on the other one run "nuttcp -t -T 5 -w 128 -v the.other.ip"
> > > >
> > >
> > > Any particular tuning of FreeBSD?
> >
> > actually my point is first to see how good or bad are the defaults.
> >
> > I have noticed that setting hw.ixgbe.max_interrupt_rate=0
> > (it is a tunable, you need to do it before loading the module)
> > improves the throughput by a fair amount (but still way below
> > line rate with 1500 byte packets).
> >
> > other things (larger windows) don't seem to help much.
> >
> > cheers
> > luigi
> > _______________________________________________
> > freebsd-current_at_freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
> >
Received on Mon Dec 05 2011 - 22:14:57 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:21 UTC