Re: MSI issues w/ Athlon64 X2 3800+ and nForce 430?

On Sat, Apr 21, 2007 at 11:44:44PM -0700, John-Mark Gurney wrote:
 > I finally upgraded to -current that supports MSI, and I appear to be
 > having issues w/ it.  I have two different ethernet cards that exhibit
 > issues on this system.  An msk PCIe based card, and a PCI em based card...
 > 
 > With MSI the msk card produced regular:
 > msk1: watchdog timeout (missed Tx interrupts) -- recovering
 > 
 > But was usable enough to push ~110meg/sec out.  When I was watching an
 > HD program streamed over http, there were regular hickups ever few
 > minutes, which I figured was due to the above messages (though none

AFAIK there is one known issue in msk(4) TSO code. You can get
corrupted IP packets while TSO is active with your msk(4). This could
explain your regular hickups as TCP may have to resend the corrupted
packets. Not sure about watchdog issues but I've received a report
for watchdog issues on CURRENT msk(4) so I have to investigate it too.
I guess it could be related with flow control(pause) packet which I
have to diagnose the root cause.

I'll disable msk(4) TSO code completely in a week if I I can't find a
workaround for the issue.

 > appeared while watching), so I decided to switch back to my em based
 > card, and w/ MSI the card is practically useless.  I get:
 > Apr 21 22:13:09 carbon kernel: em0: watchdog timeout -- resetting
 > Apr 21 22:13:09 carbon kernel: em0: link state changed to DOWN
 > Apr 21 22:13:12 carbon kernel: em0: link state changed to UP
 > Apr 21 22:14:19 carbon kernel: em0: watchdog timeout -- resetting
 > Apr 21 22:14:19 carbon kernel: em0: link state changed to DOWN
 > 
 > Which ends up making things a bit difficult to pass usable traffic over
 > the interface.
 > 
 > After disabling MSI w/:
 > hw.pci.enable_msi=0
 > in /boot/laoder.conf, things are now working again w/ em0.  I have also
 > switched back to msk1, and don't seem to be seing the watchdog timeouts

Would you setup your networks and make receiver send a pause packet to
a host with msk(4)?

 > that I was previously seeing..
 > 
 > Attached are pciconf -lv and dmesg from the box.
 > 
 > -- 
 >   John-Mark Gurney				Voice: +1 415 225 5579
 > 
 >      "All that I will do, has been done, All that I have, has not."

 > none0_at_pci0:0:0:	class=0x050000 card=0x50001458 chip=0x02f110de rev=0xa2 hdr=0x00
 >     vendor     = 'Nvidia Corp'
 >     device     = 'C51 Host Bridge'
 >     class      = memory
 >     subclass   = RAM

[...]

 > nve0: <NVIDIA nForce MCP13 Networking Adapter> port 0xdc00-0xdc07 mem 0xe9208000-0xe9208fff irq 22 at device 20.0 on pci0
 > nve0: Ethernet address 00:16:e6:1d:20:84
 > miibus3: <MII bus> on nve0
 > e1000phy3: <Marvell 88E1116 Gigabit PHY> PHY 1 on miibus3
 > e1000phy3:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto
 > nve0: using obsoleted if_watchdog interface
 > nve0: Ethernet address: 00:16:e6:1d:20:84
 > nve0: [ITHREAD]

You've got a MCP13 network adapter. I guess you could get occasional
watchdog timeouts from warm reboot from Windows. Overhauled nfe(4)
has a workdaround code that takes MAC/PHY out of power down mode
so you'd get better result with nfe(4) than nve(4).
http://people.freebsd.org/~yongari/nfe/if_nfe.c
http://people.freebsd.org/~yongari/nfe/if_nfereg.h
http://people.freebsd.org/~yongari/nfe/if_nfevar.h

-- 
Regards,
Pyun YongHyeon