Re: dev.bce.X.com_no_buffers increasing and packet loss

On Tue, Mar 09, 2010 at 11:55:30PM +0200, Ian FREISLICH wrote:
> Pyun YongHyeon wrote:
> > On Tue, Mar 09, 2010 at 03:31:55PM +0200, Ian FREISLICH wrote:
> > > Can you explain the tunables please - I'm guessing it's these:
> 
> I think I asked the wrong question.  What is a "Quick BD Chain"?

I don't know why Broadocom uses 'Quick'. It's a just buffer
descriptor chain for TX/RX.

> What relation should this number have to traffic rate.  Is there a
> maximum and what are reasonable numbers for setting this to?
> 

The maximum BD chain would be number of configured TX/RX descriptors.
And 1 would be minimum value as you have to want to get status block
for each TX/RX. Finding best value may depend on specific load.

> I set the RX as high as 512 in 64 quanta but it made little difference
> to the interrupt rate.  At times where we experience the packet
> loss and com_no_buffers increases, the interrupt rate on between 1
> and 3 of the 4 bce interfaces fell from about 3200/s to 130/s.
> 

BD chain is just one of parameters. bce(4) controllers also provide
more advanced features that fine control interrupt moderation(TX/RX
ticks). It's hard to explain all the details so you may want to
read public data sheet of bce(4).
http://www.broadcom.com/collateral/pg/NetXtremeII-PG203-R.pdf
See host coalescing registers(page 484).

> We're wondering if the switches we're using could be causing this
> problem - they're Dell PowerConnect 5448.  I've seen complaints of
> random packet loss caused by these switches on the Internet.  We
> have some new H3C 5100 series switches which we're planning on
> swapping for the Dells tomorrow to see if it makes a difference.
> 

Not sure but it does not explain increasing
dev.bce.X.com_no_buffers counter.

> Ian
> 
> --
> Ian Freislich