Re: CFT: msk(4) 64bit DMA support

On Sun, Jun 05, 2011 at 02:23:57PM -0400, David Schultz wrote:
> 85;95;0cOn Thu, May 26, 2011, YongHyeon PYUN wrote:
> > Here is a patch that implements 64bit DMA on msk(4). If you use
> > msk(4) on a system that has more than 4GB memory, please try the
> > patch at the following URL and let me know whether it works or not.
> > You need latest msk(4) in HEAD to apply the patch.
> > http://people.freebsd.org/~yongari/msk/msk.64bit.dma.diff
> > 
> > Previously msk(4) may have used bounce buffers on systems that have
> > more than 4GB memory. You can verify whether msk(4) is using bounce
> > buffers by checking the output of "sysctl hw.busdma". For instance,
> > hw.busdma.zone0.total_bounced counter would increase while network
> > operation is in progress.  If patch above works you wouldn't see
> > the counter change anymore and it would also enhance network
> > performance since it wouldn't have to copy from or to bounce
> > buffers.
> 

Sorry for late reply.

> After applying this patch, I still see total_bounced increasing:
> 	hw.busdma.zone0.total_bounced: 441
> 

Hmm, I guess it could be caused by other drivers in the system. Can
you verify whether all other drivers in the system use 64bit DMA?
I think just testing msk(4) with netperf/iperf will make it clear
(i.e. no disk access).

> Note that I have MSI disabled to work around some issues with the
> card becoming wedged:
> 	hw.pci.enable_msix=0
> 	hw.pci.enable_msi=0
> 

MSI has nothing to do with 64bit DMA.

> Possibly relevant bits of dmesg:
> 
> FreeBSD 9.0-CURRENT #4 r222717M: Sun Jun  5 12:27:07 EDT 2011
> CPU: Intel(R) Core(TM)2 Duo CPU     E8400  _at_ 3.00GHz (3000.06-MHz K8-class CPU)
>   Origin = "GenuineIntel"  Id = 0x10676  Family = 6  Model = 17  Stepping = 6
>   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>   Features2=0x8e3fd<SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1>
>   AMD Features=0x20100800<SYSCALL,NX,LM>
>   AMD Features2=0x1<LAHF>
>   TSC: P-state invariant, performance statistics
> real memory  = 8589934592 (8192 MB)
> avail memory = 8246677504 (7864 MB)
> Event timer "LAPIC" quality 400
> ACPI APIC Table: <IntelR AWRDACPI>
> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
> FreeBSD/SMP: 1 package(s) x 2 core(s)
> ioapic0: Changing APIC ID to 4
> ioapic0 <Version 2.0> irqs 0-23 on motherboard
> mskc0: <Marvell Yukon 88E8053 Gigabit Ethernet> port 0xae00-0xaeff mem 0xfdefc000-0xfdefffff irq 17 at device 0.0 on pci4
> msk0: <Marvell Technology Group Ltd. Yukon EC Id 0xb6 Rev 0x02> on mskc0
> msk0: Ethernet address: 00:01:29:a3:3c:a3
> miibus0: <MII bus> on msk0
> e1000phy0: <Marvell 88E1111 Gigabit PHY> PHY 0 on miibus0
> e1000phy0:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
> msk0: link state changed to UP