Re: Generic Kernel API

Charles Swiger writes:
 > be, except perhaps in the sound and UART drivers.  Most other FreeBSD  
 > drivers are "flat" and have a fair amount of code duplication, rather  
 > than using OO inheritance so that your fxp or dc driver inherits from  
 > a common NIC abstraction (Apple's IONetworkController ->  
 > IOEthernetController -> _device_).
 > 
 > Apple has found that using inheritance is a big win for them: "In  
 > addition, code reusability decreases the memory footprint of drivers;  
 > drivers ported from Mac OS 9, for example, have been up to 75%  
 > smaller in Mac OS X."  Of course, it's easier to say such things then  
 > to write the code, but Apple has achieved pretty good results from  
 > the IOKit.


I maintain network drivers on FreeBSD, and MacOSX (among others).  I
have found that even with all the "reusability" in IOKit, IOKit
drivers tend to have a larger footprint than FreeBSD drivers.  For
example, the file $OS/mx_ether.c{pp} provides the an interface between
the hardware and the network stack for our MX software for 4Gb/s
Myrinet adaptors:

     792 driver/freebsd/mx_ether.c
    1138 driver/macosx/mx_ether.cpp

Also, IOKit has lots of problems.  For example, there is a *big* API
bug with the way IOKit does DMA mapping for network buffers which
prevents MacOSX from doing any sort of zero-copy transmits (like
sendfile).  

The problem is that there is no analogue to bus_dmamap_unload() in
IOKit, and IOKit drivers which use IOMbufMemoryCursor() are limited to
operating on mbufs and clusters, not M_EXT external storage bufffers.
This is because MacOSX pre-pins its pool of mbufs and loads them into
the G5 IOMMU when mbuf clusters are created.  A driver will call
getPhysicalSegments() to get the DMA addresses for the mbufs he was
passed.  But getPhysicalSegments will choke on M_EXT buffers, and even
if it didn't there is no putPhysicalSegments API call to free them
from the IOMMU when the transmit is complete.

There are many other problems, such as the whole confusing
IOOutputQueue() abstraction which takes away opportunities for
parallelism and sticks a big confusing mess of goo between your driver
and ifnet.  When we were forced to move from a pure ifnet based driver
in 10.3 to an IONetworking driver in Tiger, our packets-per-second for
64 byte frames rate decreased by close to 50% even while our
underlying firmware on the NIC improved by about 25%.  I blame the
IOOutputQueue layer for this.

Lastly, IOKit itself is not 64-bit safe, primarily because the Apple
kernel is not 64-bit safe.  We have worked around this problem in the
OS-bypass side of our drivers by avoiding the entire UserClient
interface and using BSD ioctls which are much faster, and which
are 64-bit safe.

I think that supporting binary IOKit drivers as if they were another
Project Evil is a great goal.  (Maybe we could call it "Project Lesser
Evil").  But I don't think the MacOSX IOKit API is appropriate 
native API for FreeBSD.

Drew