[RFC] Patch to improve TSO limitation formula in general

Hi,

I've tested the attached patch with success and would like to have some 
feedback from other FreeBSD network developers. The problem is that the 
current TSO limitation only limits the number of bytes that can be 
transferred in a TSO packet and not the number of mbuf's.

The current solution is to have a quick and dirty custom m_dup() in the 
TX path to re-allocate the mbuf chains into 4K ones to make it simple. 
All of this hack can be avoided if the definition of the TSO limit can 
be changed a bit, like shown here:


  /*
+ * Structure defining hardware TSO limits.
+ */
+struct if_tso_limit {
+       u_int raw_value[0];     /* access all fields as one */
+       u_char frag_count;      /* maximum number of fragments: 1..255 */
+       u_char frag_size_log2;  /* maximum fragment size: 2 ** (12..16) */
+       u_char hdr_size_log2;   /* maximum header size: 2 ** (2..8) */
+       u_char reserved;        /* zero */
+};


First we need to know the maximum fragment count. Typical value is 32.
Second we need to know the maximum fragment size. Typical value is 4K.
Last we need to know of any headers that should be subtracted from the 
maximum. Hence this code is running in the fast path, I would like to 
use "u_char" for all fields and allow copy-only access as a "u_int" as 
an optimization. This avoids cludges and messing with additional header 
files.

I would like to push this patch after some more testing to -current and 
then to 10-stable hopefully before the coming 10-release, because the 
current solution is affecting performance of the Mellanox based network 
adapters in an unfair way. For example by setting the current TSO limit 
to 32KBytes which will be OK for all-2K fragments, we see a severe 
degradation in performance. Even though the hardware is fully capable of 
transmitting 16 4K mbufs.

Comments and reviews are welcome!

--HPS