Re: twa driver needs updating

Gary Palmer wrote:
> On Fri, Oct 19, 2007 at 04:02:17PM -0600, Scott Long wrote:
>> ECEG / Daniel Duerr wrote:
>>> Thank you for redirecting my message.  I've been keeping these machines 
>>> updated on RELENG_6_2 because I have a couple pre-compiled hardware 
>>> drivers I must use which I do not have kernel source for.  I just did a 
>>> csup/buildkernel last week on the latest p8 source tree and tested the 
>>> included twa driver and it was not up to date.  Would it be possible to 
>>> get the twa driver pushed back into  RELENG_6_2?
>>>
>> There are plans to get the new device support pushed back to RELENG_6 in
>> time for the 6.3 release.  I know that's not the best answer for you at
>> this moment.  What is really, truly needed here is a "driver update
>> disk" setup for the installer that allows you to load vendor-supplied
>> drivers at install time and have those drivers integrated into the
>> installed system.  Sysinstall has a clunky feature that kinda sorta
>> satisfies the first requirement, but completely falls down on the second
>> requirement, making it completely useless.  If anyone is serious about
>> sitting down and making such a feature happen, let me know.  It would be
>> an incredibly positive step towards better vendor support.  I'd like to
>> be involved because I've been in the vendor role in the past with
>> supporting linux, and I'd like to make sure that the linux mistakes
>> aren't repeated.
> 
> Since we still build a bunch of drivers into the kernel, how do you 
> propose replacing a driver that is part of GENERIC?  I think implementing
> the above feature will mean fundamentally changing the way we ship
> kernels.

One option is to change GENERIC to only include the bare minimum
components, and have everything else be a module.  You'd then have the
installer either loader every module in existence and see what attaches
and what doesn't, or have the installer and/or loader be driven by an
identification table.  The table method has been talked about for years,
and it's always been held up by disagreements and uncertainty about how
to integrate such a thing into the driver API so that it doesn't
introduce a bunch of extra work for driver writers.  There are ways to
break the impasse that I'd be happy to discuss.

A different approach is to do tricks with the runtime loader/linker so
that symbols can be overridden.  As Xin said in another email, a
mechanism already exists for drivers to get attachment priority over one
another.  The problem is letting two drivers with the same symbols into
the kernel at the same time.  Vendors have been doing hacks over the
years with renaming their symbols at compile time so that they don't
collide with the driver embedded into GENERIC, but that's painful and
clunky.  Having an override or dynamic renaming system means that
GENERIC can remain the same, eliminating all of the pain of ID tables.

Both methods have their strengths and weaknesses.  Linux, windows, and
OSX to an extent follow the first method.  I think that the second
method is intriguing, but I don't know what technical landmines might
exist.

The other big problem is getting the vendor driver onto the target
system during install, and integrating it into the build so that a
rebuild of the kernel doesn't make it suddenly go away.  Driver packages
could get bundled as regular FreeBSD packages and installed and managed
that way.  They could have some extra metadata for the installer to use
that would be ignored by the normal pkg mechanism.  The package could
also hold both binary and source bits.  The big missing piece here is
getting the source to integrate into the kernel build.  This probably
isn't too hard to deal with if you accept that such drivers will only
be built as modules and not as static parts of the kernel.  There are
already vectors in the build system to build specific modules in
specific ways.  Extending this would be pretty trivial.

What I don't want to see is this work get derailed by grand proposals.
The last time this topic was discussed, it died with a promise to
radically change the kernel build and linking system into one where the
kernel was just a specially linked set of modules (which would take
care of the caveat at the end of the previous paragraph).  Proposals
like that are great, but they shouldn't be prerequisites for the larger
task and the larger goal.

What is don't like about the Redhat way:

- The driver ID table is focused around just PCI.  It has no provisions
for something like USB.  It's also Redhat specific and separate from the
kernel, making it a PITA for vendors to deal with.
- Only SCSI and NET drivers can be loaded during install
- Kernel symbols are versioned based on a hash of the kernel version and
certain options.  Multiple binary versions of the driver then need to be
built and distributed to handle options like SMP.  This is absolutely
maddening for vendors to deal with.
- Only limited scripting is available to the driver package.  The
package isn't integrated into the RPM system, making it unmanageable.

There's probably a dozen other things that I absolutely hated as a
vendor about Redhat, but I've almost certainly repressed those bad
memories =-)

This task can be broken into parts that can be worked on independently.
Bundling/packaging, build system integration, installer integration, and
kernel linking can all be separate tasks that are worked on in parallel.

Scott