Re: any place to look at for PCI-express performance issues ?

On Thu, Jun 09, 2011 at 09:11:51AM -0400, John Baldwin wrote:
> On Wednesday, June 08, 2011 11:59:52 pm Luigi Rizzo wrote:
> > hi,
> > during my tests with netmap with 10Gbit cards (82599, dual port),
> > i notice that a motherboard with an AMD 880G chipset
> > is performing significantly worse than an intel based one.
> > In both cases the NIC is mounted on a 16x PCIe slot,
> > and in both cases the driver reports the use 5Gb/4x per port.
> > 
> > On the intel i reach easily 14.88Mpps, on the AMD the card tops
> > at 1.8Mpps, and is not CPU limited (changing dev.cpu.0.freq does not change
> > the throughput).
> > Disabling flow control does not help (and in any case
> > the other end of the link is the same), and since
> > I am using the same picobsd image (based on FreeBSD/i386
> > head w/ my netmap code) i suspect that the difference in
> > performance has to do with the PCIe controller.

just for the records: the AMD motherboard works fine and can reach
14.88Mpps, i was just doing a couple of mistakes in my AMD tests,
including the use of a slot with 16x form factor but only 4 lanes
connected.

This said, the i7-870 is about twice as fast as the Athlon II X4-635
in generating packets for the same clock speed.

I think the different cache size might have some impact on the
result given the Athlon has no L3 cache and the test program surely
overflows the 512k L2 cache (i am using a total of 8k packet buffers,
touching 64 bytes each for the payload, plus 24 bytes each for
descriptors).

Unfortunately at these speeds even small things matter a lot!

cheers
luigi