Re: High rate traffic silence an em interface.

On Sat, 25 Sep 2004, Shunsuke SHINOMIYA wrote:

>  traffic over output interface's transmission rate silence an em
> interface with 5.3-BETA5. 
> 
>  I configured a P4 with HTT box with two em interfaces for a router, one
> interface is set to 100BaseTX, the other is set to 10BaseT. 
> 
>  And I sent the IPv4 11Mbps(only 1Mbps exceed 10Mbps) traffic for 10
> seconds from 100BaseTX side to 10BaseT side by smartbits, most of
> packets dropped and this measurement terminated with failure. 
>  Then I did ping to a host over 10baseT side at the box, ping outputted
> with "No buffers space avilable". 

I'm seeing this also.  For me, the threshold to trigger the problem is
somewhere between 50Kpps and 250Kpps, depending on factors I can't
identify.  However, the box I'm running on also has some interesting
interrupt configuration issues, so I'm wondering if what I'm looking at is
a driver race condition.  It seems not to be related to Giant over the
network stack -- I see the weding with and without debug.mpsafenet=1. 

I saw Sam Leffler also report a similar problem today.  Bosko and I have
been seeing this if_em problem on some test hardware we have been using
for several months.  Bosko tried updating to the latest version of the
driver from Intel's web site, and it appeared to make the problem go away. 
However, the version on Intel's web site doesn't have busdma support or
fine-grained locking, so if it is a race condition, maybe their version of
the driver doesn't trigger it because it's doing these things differently
(i.e., might still exist there).

You might try using the netrate tool I committed to
src/tools/tools/netrate to try and figure out the threshold transmission
level necessary to trigger the problem.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert_at_fledge.watson.org      Principal Research Scientist, McAfee Research


> 
> >> ping 10.1.1.3
> >PING 10.1.1.3 (10.1.1.3): 56 data bytes
> >ping: sendto: No buffer space available
> >^C
> >--- 10.1.1.3 ping statistics ---
> >1 packets transmitted, 0 packets received, 100% packet loss
> 
>  I found two methods for recovery. One is up & down the interface of
> 10BaseT side, the other (strange?) one is ping6 to the host over 10baseT
> side like "ping6 ff02::1%em1".
> 
>  "Tx Descriptors not avail1" counter by hw.em1.debug_info increased.
> Another counters kept zero.
> 
> > em1: Adapter hardware address = 0xc3dc6b34
> > em1:CTRL  = 0x40f01849
> > em1:RCTL  = 0x8002 PS=(0x8402)
> > em1:tx_int_delay = 0, tx_abs_int_delay = 0
> > em1:rx_int_delay = 0, rx_abs_int_delay = 0
> > em1: fifo workaround = 0, fifo_reset = 0
> > em1: hw tdh = 132, hw tdt = 132
> > em1: Num Tx descriptors avail = 256
> > em1: Tx Descriptors not avail1 = 6430
> > em1: Tx Descriptors not avail2 = 0
> > em1: Std mbuf failed = 0
> > em1: Std mbuf cluster failed = 0
> > em1: Driver dropped packets = 0
> 
> 
>  Is this a peculiar problem just with me?
> 
> -- 
> Shunsuke SHINOMIYA <shino_at_fornext.org>
> 
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>