Re: Problems with jumbo frames on nfe

On Wed, May 20, 2009 at 03:07:23PM +0900, Pyun YongHyeon wrote:
> On Tue, May 19, 2009 at 03:14:32PM +0200, Pascal Braun wrote:
> > Pyun YongHyeon wrote:
> > >On Tue, May 19, 2009 at 11:31:02AM +0200, Pascal Braun wrote:
> > >>Hi,
> > >>
> > >>I'm currently testing jumbo frames on a Sunfire 4540 running with 
> > >>FreeBSD 8-Current (build on April 8th). While using the nfe driver I'm 
> > >>having some unexpected problems if i try to bring up the interface with 
> > >>MTU sizes greater than 1970 bytes.
> > >>
> > >>The error message (in dmesg) is:
> > >>nfe2: initialization failed: no memory for rx buffers
> > >>
> > >
> > >It means you've run out of jumbo clusters. Check the output of
> > >"netstat -m" and see how many jumbo cluster requests were denied.
> > 
> > 771/1149/1920 mbufs in use (current/cache/total)
> > 770/680/1450/25600 mbuf clusters in use (current/cache/total/max)
> > 770/510 mbuf+clusters out of packet secondary zone in use (current/cache)
> > 0/9/9/12800 4k (page size) jumbo clusters in use (current/cache/total/max)
> > 0/0/0/16384 9k jumbo clusters in use (current/cache/total/max)
> > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
> > 1732K/1683K/3416K bytes allocated to network (current/cache/total)
> > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> > 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
> > 0/0/0 sfbufs in use (current/peak/max)
> > 0 requests for sfbufs denied
> > 0 requests for sfbufs delayed
> > 0 requests for I/O initiated by sendfile
> > 0 calls to protocol drain routines
> > 
> > There are no denied requests. MTU size was about 1800.
> > But I have to add, that i cant even get the interface up if the mtu size 
> > is above 1970.
> > 
> > >>Does anyone have any ideas how to get jumbo frames working?
> > >>
> > >
> > >How about increasing 9K jumbo clusters(kern.ipc.nmbjumbo9) with
> > >sysctl(8)?
> > 
> > sunfire# sysctl -a | grep jumbo
> > kern.ipc.nmbjumbo16: 3200
> > kern.ipc.nmbjumbo9: 16384
> > kern.ipc.nmbjumbop: 12800
> > 
> > I increased kern.ipc.nmbjumbo9 to 16384 (was 6400) but that didn't seem 
> > to help. Do you have another idea?
> > 
> Ok, would you try attached patch?
> 

FYI: I've committed the patch(r192706).