Re: [RFC] Allow m_dup() to use JUMBO clusters

On Mon, Jul 07, 2014 at 10:12:07AM +0200, Hans Petter Selasky wrote:
> Hi,
> 
> I'm asking for some input on the attached m_dup() patch, so that 
> existing functionality or dependencies are not broken. The background 
> for the change is to allow m_dup() to defrag long mbuf chains that 
> doesn't fit into a specific hardware's scatter gather entries, typically 
> when doing TSO.
> 
> In my case the HW limit is 16 entries of length 4K for doing a 64KByte 

I wonder how HW can handle a full-sized TSO packet(64KB + Ethernet
header + VLAN tag).

> TSO packet. Currently m_dup() is at best producing 32 entries of each 2K 
> for a 64Kbytes TSO packet.
> 
> By allowing m_dup() to get JUMBO clusters when allocating mbufs, we 
> avoid creating a new function, specific to the hardware, to defrag some 
> rare-occurring very long mbuf chains into a mbuf chain below 16 entries.
> 

I think m_dup() was used to get a copy of writable mbuf chains.  If
m_dup() starts to allocate jumbo mbufs it will eventually fail on
long running boxes.  This will break firewall(ipfw divert, pf/ipf
dup-to) rules and several ethernet drivers.

I don't know how many TSO requests could be queued by HW but if the
number is very small, the driver may be able to pre-allocate that
number of buffers (N * (64KB + Ethernet header + VLAN tag)) in
driver.  Upper stack will almost always generate more than 16 mbufs
for TSO packets. When driver knows the length of mbuf chain of TSO
packet is more than 16, you can copy the mbuf chain to the
pre-allocated buffer.

I recall I didn't implement TSO on txp(4) because the firmware of
txp(4) controller does not support more than 16 fragment
descriptors.