new usb stack (was Re: FreeBSD Status Report Fourth Quarter 2006)

[i avoid the cross-post to hackers as my reply mostly affects
-current and -usb. hope i don't miss anyone interested. I am also
Bcc-ing a few people who might have something to say on the subject]

On Tue, Jan 16, 2007 at 11:52:37PM +0100, Max Laier wrote:
...
> FreeBSD Status Report

thanks all for the long and interesting report, i see lot of
stuff going on.

I have a couple of usb-related comments:

> New USB Stack
> 
>    URL:
>    http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/projects
>    /usb/src/sys/dev/usb
>    URL: http://www.turbocat.net/~hselasky/usb4bsd
> 
>    Contact: Hans Petter Sirevaag Selasky <hselasky_at_FreeBSD.org>
> 
>    During the last three months there has not been so much activity in
>    the USB project. Some regression issues have been reported and fixed.
>    Bernd Walter reports that he has got the new USB stack working on ARM
>    processors with some minor tweaks. Markus Brueffer reports that he is
>    working on the USB HID parser and support. A current issue with the
>    new USB stack is that the EHCI driver does not work on the Sparc64
>    architecture. If someone has got a Sparc64 with FreeBSD 7-CURRENT on
>    and can lend the USB project the root password, a serial console and a
>    USB test device, for example a USB memory stick, that would be much
>    appreciated. Another unresolved issue is that the ural(4) USB device
>    driver does not always work. This is currently being worked on.
> 
>    If you want to test the new USB stack, check out the USB perforce tree
>    or download the SVN version of the USB driver from my USB homepage. At
>    the moment the tarballs are a little out of date.
> 
>    Ideas and comments with regard to the new USB API are welcome at
>    freebsd-usb_at_FreeBSD.org .

I just happen to have spent some time on the USB stack trying to
put together support for some webcams i have. Apart from the details
on this specific work, which i started summarising at

	http://info.iet.unipi.it/~luigi/FreeBSD/usb-cameras.html

there are a few things that i would like to say/ask:

1. Migration strategy to the new usb stack

   One of the reasons to move to the new usb stack that Hans mentions above,
   is the lack of high speed isochronous support in the 'old' usb stack
   (i.e. the one in HEAD and RELENG_6 and below)
   I am particularly concerned about because it has to do with
   camera support.
   
   One issue with the new stack, however, is the different kernel API which
   is not backward compatible, and, as i discussed with Hans at length,
   this could be a significant obstacle to its adoption as it basically
   cuts support for third party drivers.
	The obvious solution to this problem is building an emulation
   layer on top of the new stack to offer a backward compatible API
   to old style drivers. This would serve both as a tool to support
   re-building of old-style drivers, and as an indirect source of
   documentation for adapting drivers to the new style.
	Building this emulation layer poses some difficulties,
   but in a relatively long phone call with Hans tonight we probably
   came up with a reasonable plan to solve the locking issues that existed
   in his previous implementation of the compatibility layer (removed
   some time ago because of these locking issues, you can see some detail
   in perforce http://perforce.freebsd.org/changeView.cgi?CH=107765 ).
	I hope Hans will follow up with more details, but i am confident
   that this approach will provide us with a suitable upgrade path
   that does not cost us regressions.

2. Linux compatibility layer.

   The discussion, as well as my recent work on webcams and other work
   from Raaf on DVB drivers (see http://raaf.atspace.org/dvbusb/index.html;
   and Raaf is in Bcc so he may comment if he likes)
   raised another issue, namely a linux compatibility layer.

   The linux community has developed a relatively large set of drivers
   for USB devices, for many sort of devices which we do not support
   at the moment e.g. webcams, DVB receivers, even 802.11 cards.
	Because a lot of these driver are built by reverse-engineering,
   the code tends to be on the obfuscated side, and doing a 'clueful'
   rewrite is typically not a viable option unless one has plenty of
   time to dedicate to the task. What one ends up doing is, instead,
   is a mechanical conversion, #ifdef'ing out the glue module and device
   hooks that gets replaced with FreeBSD equivalents, and then trying to
   translate the linux kernel API into more-or-less equivalent FreeBSD
   code. As i said this is a mechanical conversion, and it would be
   a tremendous help to have a linux-compatibility layer (library + macros)
   to support this work at least partially.

   More than comments like "yeah it would be great" i am wondering if
   there is any work around already done on this area. Surely people
   who ported linux drivers to FreeBSD have some clue, and maybe
   their set of macros/working notes used in the process. I have
   done something like this myself.

   I would be grateful if people interested in this could provide their
   input and/or contact me so we can hopefully set up a small project
   to implement such a compatibility layer. Part of it is generic, e.g.
   the module glue, the basic kernel services such as *malloc(), printk()
   and so on; part of it is usb-specific e.g. providing an emulation of
   linux mechanism for using the usb subsystems, e.g. urb's and so on.
   Finally, part of it could be related to the network subsystem as
   a number of devices we might be interested in are the various
   802.11 USB dongles, none of them is currently supported on FreeBSD.

Feedback welcome

	cheers
	luigi