Re: datapoints on 10G throughput with TCP ?

On 06.12.2011 22:06, Luigi Rizzo wrote:
> On Tue, Dec 06, 2011 at 07:40:21PM +0200, Daniel Kalchev wrote:
>> I see significant difference between number of interrupts on the Intel and the AMD blades. When performing a test between the Intel and AMD blades, the Intel blade generates 20,000-35,000 interrupts, while the AMD blade generates under 1,000 interrupts.
>>
>
> Even in my experiments there is a lot of instability in the results.
> I don't know exactly where the problem is, but the high number of
> read syscalls, and the huge impact of setting interrupt_rate=0
> (defaults at 16us on the ixgbe) makes me think that there is something
> that needs investigation in the protocol stack.
>
> Of course we don't want to optimize specifically for the one-flow-at-10G
> case, but devising something that makes the system less affected
> by short timing variations, and can pass upstream interrupt mitigation
> delays would help.

I'm not sure the variance is only coming from the network card and
driver side of things.  The TCP processing and interactions with
scheduler and locking probably play a big role as well.  There have
been many changes to TCP recently and maybe an inefficiency that
affects high-speed single sessions throughput has crept in.  That's
difficult to debug though.

-- 
Andre