Re: High Network Perfomance

On Fri, Aug 05, 2011 at 06:34:54PM +0300, Andriy Gapon wrote:
> on 05/08/2011 18:23 Vlad Galu said the following:
> > Netmap's scope may be narrow, but it's a great alternative to
> > proprietary implementations that are provided by only a handful of
> > vendors and tied to their hardware. I don't think Luigi foresees lots
> > of users either, but having it in the base system is a lot better than
> > not having it at all, IMHO.
> 
> No doubt.
> 
> On the other hand, it was a little bit misleading of Luigi to suggest netmap to a
> user who merely asked about tuning FreeBSD network (routing/firewall) performance :-)

Sure, i took the chance to advertise the product a bit :)

But the original poster essentially wanted to "go as fast as i can"
and any tweaks that one can do on the system won't bring him much
further than the 500Kpps he is seeing. Perhaps 1Mpps. Not too exciting.

> netmap can no doubt be useful, but perhaps let's not overhype it before it
> actually proves itself on practical tasks.  Or has it already?  I might have
> missed that.

Its the latter. The libpcap-over-netmap code has been there for
a while, but documentation was not completely up to date so
we are both at fault.
 
I just updated the webpage with links to another paper reporting
performance of the libpcap emulation library, Click and OpenvSwitch.

There is also a new set of slides for a talk i am going to give at
various locations in the Bay Area next week (if you are around,
email me privately for details)

Our libpcap makes netmap immediately available for basically
all pcap clients (sure, you might need a bit of tinkering,
and some apps could have their own speed issues -- see our study
on Click and OpenvSwitch).
Click userspace now runs (on FreeBSD+netmap) as fast or better than
the in-kernel linux version.  Which means that all research
and prototypes that were bound to Linux because of this
reason, now could consider switching platform.

Note that having netmap does not prevent the existing stack from
working. I have designed the system in a way that allows 
incremental improvement/replacement of the components.

I am working on bringing outside the kernel ipfw+dummynet
(relatively straightforward, i did it already once a couple of years ago),
and then the routing tables.

netgraph could be another candidate (though i dont know
how much work it involves; netgraph is very similar to Click,
and the latter might have a richer set of elements).
 
For TCP (and high speed TCP) i don't have a clear view on
what are the bottlenecks, but with 1500-byte MTU your pps rates
are 20 times lower, so the problem is entirely different and
saving 400ns per packet does help, but not as much as when
packets arrive every 70ns.

cheers
luigi
-----------------------------------------+-------------------------------
  Prof. Luigi RIZZO, rizzo_at_iet.unipi.it  . Dip. di Ing. dell'Informazione
  http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
  TEL      +39-050-2211611               . via Diotisalvi 2
  Mobile   +39-338-6809875               . 56122 PISA (Italy)
-----------------------------------------+-------------------------------