Re: dev.bce.X.com_no_buffers increasing and packet loss

"David Christensen" wrote:
> > Pyun YongHyeon wrote:
> > > On Wed, Mar 10, 2010 at 02:45:47PM -0800, David Christensen wrote:
> > > > The bce(4) hardware supports a linked list of pages for RX buffer=20
> > > > descriptors.  The stock build supports 2 pages (RX_PAGES) with a=20
> > > > total of 511 BD's per page.  The hardware can support a=20
> > maximum of=20
> > > > 64K BD's but that would be an unnecessarily large amount of mbufs=20
> > > > for an infrequent problem.
> >=20
> > I think that depends on how you define infrequent.  Our use=20
> > case is a largish core router.  It's highly likely that we'll=20
> > see this again and again in various packet storms on our network.
> >=20
> 
> Are the packet storms always always from the same host or do they come
> from multiple hosts?  The hardware supports RSS which can spread the
> network load across multiple receive queues and multiple CPU cores, but
> only when the traffic is spread across several hosts.  (The current
> bce(4) driver doesn't include support for RSS.)  If a storm of
> small frames comes from a single host then almost all adapters will be
> challenged to handle the flow.

In this case the storm only involved 2 hosts.  While it's an
exceptional circumstance it isn't unusual in our environment (core
router in a datacenter).  Fortuately we controlled both machines
in this instance.  Perhaps if the load is spread across more CPUs,
then perhaps only those flows unlucky to hash to the CPU handling
the storm will be degraded.  That is a marginally better situation
than all flows being degraded.  From the sounds of it RSS isn't the
cure for this particular situation, but may improve performance in
general.

It does sound like reworking the buffer will solve the problem.
Perhaps having a 2 recieve rings so that once one ring is available
for processing, the other ready filled and clear ring can be used
for recieving frames.

Ian

--
Ian Freislich