Re: vge traffic problem

On Sun, Jan 03, 2010 at 06:55:54AM -0800, David Ehrmann wrote:
> I reported this problem before, but I have a specific, reproducible 
> example, now.  I'm running 8.0 RC1.  I'm trying to fetch the source to 
> upgrade to 8.0, but this happens:
> 
> # csup stable-supfile
> Connected to 72.233.193.64
> Receiver: Connection reset by peer
> Will retry at 22:39:31
> 
> tcpdump output of that: http://pastebin.com/m16978f3 (there seem to be a 
> lot of duplicate acks being sent).
> 
> I tried with a different server, and I tried several times, but 
> generally, the same thing, though I do occasionally get lucky.
> 
> I have a vr interface on my motherboard, so I gave it an IP address in a 
> different subnet and changed the default gateway to a NAT box with the 
> same default gateway as this box used to have (so I'm using the same 
> internet connection).  Suddenly, it works.  Reproducibly.
> 
> I also have issues with samba, but putting that aside, netstat -s -p tcp 
> reported this:
> 
> tcp:
> ...
>        289024537 packets received
>                116305350 acks (for 1099476725 bytes)
>                3119202 duplicate acks
> 
> That's a duplicate ack received for every 37 unique acks.  Seems pretty 
> high for traffic on a small LAN, but I could be wrong.  I checked the 
> cable; it is cat5e or cat 6.  I also tried plugging the vge port into a 
> 10/100 mbps switch, but csup still failed.
> 

I vaguely guess it comes from long standing vge(4) bug. If a
sending frame is composed of exactly 7 fragments, vge(4) used to set
to 8 because the hardware expect # fragments + 1 in TX descriptor.
However the bit fields that represents the information has 3 bits
such that setting it to 8 yields 0 in this field and this seems to
confuse controller.

> Ideas?  I saw that there were a few recent changes to vge, but I'm not 
> sure if they're related to this.

I think I fixed all known vge(4) issues in HEAD. vge(4) in HEAD has
new hardware based interrupt moderation, hardware MAC statistics
support and WOL as well as fixing long standing bugs. Would you try
latest vge(4) in HEAD?(Just download if_vge.c, if_vgereg.h and
if_vgevar.h from HEAD and rebuild it). If your vge(4) controller
use IC Plus IP1001PHY you may also want to download ip1000phy.c in
HEAD to reliably establish 1000baseT link after waking up from
WOL/resume.

There is one unresolved TX performance issue though. I couldn't get
more than 800Mbps on TCP bulk transfers. All PCIe controllers I'm
aware of can get more than 900Mbps so there might be some missing
bits in vge(4).