Re: Problems with jumbo frames on nfe

On Tue, May 19, 2009 at 03:14:32PM +0200, Pascal Braun wrote:
> Pyun YongHyeon wrote:
> >On Tue, May 19, 2009 at 11:31:02AM +0200, Pascal Braun wrote:
> >>Hi,
> >>
> >>I'm currently testing jumbo frames on a Sunfire 4540 running with 
> >>FreeBSD 8-Current (build on April 8th). While using the nfe driver I'm 
> >>having some unexpected problems if i try to bring up the interface with 
> >>MTU sizes greater than 1970 bytes.
> >>
> >>The error message (in dmesg) is:
> >>nfe2: initialization failed: no memory for rx buffers
> >>
> >
> >It means you've run out of jumbo clusters. Check the output of
> >"netstat -m" and see how many jumbo cluster requests were denied.
> 
> 771/1149/1920 mbufs in use (current/cache/total)
> 770/680/1450/25600 mbuf clusters in use (current/cache/total/max)
> 770/510 mbuf+clusters out of packet secondary zone in use (current/cache)
> 0/9/9/12800 4k (page size) jumbo clusters in use (current/cache/total/max)
> 0/0/0/16384 9k jumbo clusters in use (current/cache/total/max)
> 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
> 1732K/1683K/3416K bytes allocated to network (current/cache/total)
> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
> 0/0/0 sfbufs in use (current/peak/max)
> 0 requests for sfbufs denied
> 0 requests for sfbufs delayed
> 0 requests for I/O initiated by sendfile
> 0 calls to protocol drain routines
> 
> There are no denied requests. MTU size was about 1800.
> But I have to add, that i cant even get the interface up if the mtu size 
> is above 1970.
> 
> >>Does anyone have any ideas how to get jumbo frames working?
> >>
> >
> >How about increasing 9K jumbo clusters(kern.ipc.nmbjumbo9) with
> >sysctl(8)?
> 
> sunfire# sysctl -a | grep jumbo
> kern.ipc.nmbjumbo16: 3200
> kern.ipc.nmbjumbo9: 16384
> kern.ipc.nmbjumbop: 12800
> 
> I increased kern.ipc.nmbjumbo9 to 16384 (was 6400) but that didn't seem 
> to help. Do you have another idea?
> 
Ok, would you try attached patch?