Re: [RFC] Patch to add Software/Generic Segmentation Offload (GSO) support in FreeBSD

Hi Ryan,
in gso_dispatch(), I put the "eh_len" parameter in order to have the
offset of the L3 header. In this way, if someone adds QinQ support,
just call gso_dispatch() with the right length of the MAC header.
During the execution the GSO, the MAC header is simply copied
as it is in each new segment.

Instead, for the vxlan support, we can define a new entries in gso_type,
define a new "gso_functions" to properly handle these types of packets
and mark the packet in the network stack with the correct GSO type.
For now we used only 4 bit to encode the gso_type in m_pkthdr.csum_flags,
but, in the future, we can use more bit or a specific field in the m_pkthdr.

Your suggestions are very good, but I tried to make a software TSO,
modifying as little as possible the network stack.

Thanks,
Stefano




2014-09-18 20:50 GMT+02:00 Ryan Stone <rysto32_at_gmail.com>:

> On Wed, Sep 17, 2014 at 4:27 AM, Stefano Garzarella
> <stefanogarzarella_at_gmail.com> wrote:
> > Much of the advantage of TSO comes from crossing the network stack only
> > once per (large) segment instead of once per 1500-byte frame.
> > GSO does the same both for segmentation (TCP) and fragmentation (UDP)
> > by doing these operations as late as possible.
>
> My initial impression is that this is a layering violation.  Code like
> this gives me pause:
>
> + eh = mtod(m, struct ether_vlan_header *);
> + if (eh->evl_encap_proto == htons(ETHERTYPE_VLAN)) {
> + eh_len = ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN;
> + } else {
> + eh_len = ETHER_HDR_LEN;
> + }
> +
> + return gso_dispatch(ifp, m, eh_len);
>
> If someone adds QinQ support, this code must be updated.  When vxlan
> support comes in, we must update this code or else the outer UDP
> packet gets fragmented instead of the inner TCP payload being
> segmented.  As more tunneling protocols get added to FreeBSD, the
> dispatch code for GSO gets uglier and uglier.
>
> It seems to me that the real problem that we are trying to solve is a
> lack of batching in the kernel.  Currently the network stack operates
> on the mbuf (packet) boundary.  It seems to me that we could introduce
> a "packet group" concept that is guaranteed to have the same L3 and L2
> endpoint.  In the transmit path, we would initially have a single
> (potentially oversized) packet in the group.  When TCP segments the
> packet, it would add each packet to the packet group and pass it down
> the stack.  Because we guarantee that the endpoints are the same for
> every packet in the group, the L3 code can do a single routing table
> lookup and the L2 code can do a single l2table lookup for the entire
> group.
>
> The disadvantages of packet groups would be that:
> a) You have touch a lot more code in a lot more places to take
> advantage of the concept.
> b) TSO inherently has the same layering problems.  If we're going to
> solve the problem for tunneling protocols then GSO might well be able
> to take advantage of them.
>



-- 
*Stefano Garzarella*
stefano.garzarella_at_gmail.com