Re: dev.bce.X.com_no_buffers increasing and packet loss

On Fri, Mar 05, 2010 at 10:20:18PM +0200, Ian FREISLICH wrote:
> Pyun YongHyeon wrote:
> > 
> > Would you show me the output of dmesg(bce(4)/brgphy(4) only) and
> > the output of "pciconf -lcbv" for the controller?
> 
> [firewall1.jnb1] ~ # egrep "bce|brgphy" /var/run/dmesg.boot 
> bce0: <Broadcom NetXtreme II BCM5708 1000Base-T (B2)> mem 0xe6000000-0xe7ffffff irq 72 at device 0.0 on pci4
> miibus0: <MII bus> on bce0
> brgphy0: <BCM5708C 10/100/1000baseTX PHY> PHY 1 on miibus0
> brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
> bce0: Ethernet address: 00:1e:c9:4a:33:b9
> bce0: [ITHREAD]
> bce0: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C (4.0.3); Flags (MSI|MFW); MFW (ipms 1.6.0)
> bce1: <Broadcom NetXtreme II BCM5708 1000Base-T (B2)> mem 0xe8000000-0xe9ffffff irq 75 at device 0.0 on pci6
> miibus1: <MII bus> on bce1
> brgphy1: <BCM5708C 10/100/1000baseTX PHY> PHY 1 on miibus1
> brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
> bce1: Ethernet address: 00:1e:c9:4a:33:bb
> bce1: [ITHREAD]
> bce1: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C (4.0.3); Flags (MSI|MFW); MFW (ipms 1.6.0)
> bce2: <Broadcom NetXtreme II BCM5708 1000Base-T (B2)> mem 0xea000000-0xebffffff irq 33 at device 0.0 on pci8
> miibus2: <MII bus> on bce2
> brgphy2: <BCM5708C 10/100/1000baseTX PHY> PHY 1 on miibus2
> brgphy2:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
> bce2: Ethernet address: 00:1e:4f:fb:cf:c5
> bce2: [ITHREAD]
> bce2: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C (4.0.3); Flags (MSI|MFW); MFW (ipms 1.6.0)
> bce3: <Broadcom NetXtreme II BCM5708 1000Base-T (B2)> mem 0xec000000-0xedffffff irq 37 at device 0.0 on pci10
> miibus3: <MII bus> on bce3
> brgphy3: <BCM5708C 10/100/1000baseTX PHY> PHY 1 on miibus3
> brgphy3:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
> bce3: Ethernet address: 00:1e:4f:fb:cf:c7
> bce3: [ITHREAD]
> bce3: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C (4.0.3); Flags (MSI|MFW); MFW (ipms 1.6.0)
> 
> bce0_at_pci0:4:0:0:        class=0x020000 card=0x02231028 chip=0x164c14e4 rev=0x12 hdr=0x00
>     vendor     = 'Broadcom Corporation'
>     device     = 'Broadcom NetXtreme II Gigabit Ethernet Adapter (BCM5708)'
>     class      = network
>     subclass   = ethernet
>     bar   [10] = type Memory, range 64, base 0xe6000000, size 33554432, enabled
>     cap 07[40] = PCI-X 64-bit supports 133MHz, 512 burst read, 8 split transactions
>     cap 01[48] = powerspec 2  supports D0 D3  current D0
>     cap 03[50] = VPD
>     cap 05[58] = MSI supports 1 message, 64 bit enabled with 1 message
> bce1_at_pci0:6:0:0:        class=0x020000 card=0x02231028 chip=0x164c14e4 rev=0x12 hdr=0x00
>     vendor     = 'Broadcom Corporation'
>     device     = 'Broadcom NetXtreme II Gigabit Ethernet Adapter (BCM5708)'
>     class      = network
>     subclass   = ethernet
>     bar   [10] = type Memory, range 64, base 0xe8000000, size 33554432, enabled
>     cap 07[40] = PCI-X 64-bit supports 133MHz, 512 burst read, 8 split transactions
>     cap 01[48] = powerspec 2  supports D0 D3  current D0
>     cap 03[50] = VPD
>     cap 05[58] = MSI supports 1 message, 64 bit enabled with 1 message
> bce2_at_pci0:8:0:0:        class=0x020000 card=0x1f121028 chip=0x164c14e4 rev=0x12 hdr=0x00
>     vendor     = 'Broadcom Corporation'
>     device     = 'Broadcom NetXtreme II Gigabit Ethernet Adapter (BCM5708)'
>     class      = network
>     subclass   = ethernet
>     bar   [10] = type Memory, range 64, base 0xea000000, size 33554432, enabled
>     cap 07[40] = PCI-X 64-bit supports 133MHz, 512 burst read, 8 split transactions
>     cap 01[48] = powerspec 2  supports D0 D3  current D0
>     cap 03[50] = VPD
>     cap 05[58] = MSI supports 1 message, 64 bit enabled with 1 message
> bce3_at_pci0:10:0:0:       class=0x020000 card=0x1f121028 chip=0x164c14e4 rev=0x12 hdr=0x00
>     vendor     = 'Broadcom Corporation'
>     device     = 'Broadcom NetXtreme II Gigabit Ethernet Adapter (BCM5708)'
>     class      = network
>     subclass   = ethernet
>     bar   [10] = type Memory, range 64, base 0xec000000, size 33554432, enabled
>     cap 07[40] = PCI-X 64-bit supports 133MHz, 512 burst read, 8 split transactions
>     cap 01[48] = powerspec 2  supports D0 D3  current D0
>     cap 03[50] = VPD
>     cap 05[58] = MSI supports 1 message, 64 bit enabled with 1 message
> 

Thanks for the info. Frankly, I have no idea how to explain the
issue given that you have no heavy load.
I have a bce(4) patch which fixes a couple of bus_dma(9) issues as
well as fixing some minor bugs. However I don't know whether the
patch can fix the RX issue you're suffering from. Anyway, would you
give it try the patch at the following URL?
http://people.freebsd.org/~yongari/bce/bce.20100305.diff
The patch was generated against CURRENT and you may see a message
like "Disabling COAL_NOW timedout!" during interface up. You can
ignore that message.