RE: dev.bce.X.com_no_buffers increasing and packet loss

> > > > It's been running for about 1:23 on the patched driver. 
>  I'm still 
> > > > seeing the com_no_buffers increase:
> > > > 
> > > > [firewall2.jnb1] ~ # sysctl dev.bce |grep com_no_buffers
> > > > dev.bce.0.com_no_buffers: 5642
> > > > dev.bce.1.com_no_buffers: 497
> > > > dev.bce.2.com_no_buffers: 6260612
> > > > dev.bce.3.com_no_buffers: 4871338
> > > > 
> > > 
> > > Still have no idea why these counters are increasing here.
> > > Actually the counter is read from scratch pad of completion 
> > > processor. The datasheet does not even document the counter.
> > > Maybe david know better what's happening here(CCed).
> > > 
> > > > Interupt rate is down now, at about 3500 per second per 
> interface.
> > > > 
> > > > Interestingly setting net.inet.ip.fastforwarding=0 reduces CPU 
> > > > consumption from 25% to 9% and less packet loss.
> > 
> > The com_no_buffers statistic comes from firmware and indicates how 
> > many times a valid frame was received but could not be 
> placed into the 
> > receive chain because there were no available RX buffers.  The 
> > firmware will then drop the frame but that dropped frame won't be 
> > reflected in any of the hardware based statistics.
> > 
> 
> Yeah, but the question is why bce(4) has no available RX buffers.
> The system has a lot of available mbufs so I don't see the 
> root cause here.

What's the traffic look like?  Jumbo, standard, short frames?  Any 
good ideas on profiling the code?  I haven't figured out how to use
the CPU TSC but there is a free running timer on the device that
might be usable to calculate where the driver's time is spent.

Dave