netmap: extension to store user data per packet/slot?

Hi Luigi,
hi all,

so I was running into logistics issues with netmap(4)
with regard to zero-copy and redirection through pipes:
working on a load-balancing framework revealed that it
is very hard to track a packet's origins to later move
it onward to the respective outgoing interface, be it
another device or the host stack.

Long story short: user data needs to be stored for the
packet buffer or slot.

There are three ways that I can see so far:

(1) Allocate a netmap pipe pair for each interface,
    in case of transparent mode also a pipe for the
    host stack each.  That's a lot of pipes and
    most likely insane, but it won't extend the ABI.

(2) Store the additional data in the actual buffer.
    That is sort of ok, but seems sluggish WRT cache
    behaviour -- maybe the buffer won't be read but
    it needs to be written.  Sure, we can store it at
    the end, but there already resides the packet
    timestamp if enabled (if I recall correctly).
    Wouldn't extend the ABI per se, but might collide
    with the timestamping....

(3) Make room in struct netmap_slot itself like this:

diff --git a/sys/net/netmap.h b/sys/net/netmap.h
index 15ebf73..d0a9c0e 100644
--- a/sys/net/netmap.h
+++ b/sys/net/netmap.h
_at__at_ -147,6 +147,7 _at__at_ struct netmap_slot {
        uint16_t len;           /* length for this slot */
        uint16_t flags;         /* buf changed, etc. */
        uint64_t ptr;           /* pointer for indirect buffers */
+       uint64_t userdata;      /* reserved storage for caller */
 };

It could also be broken down in two fields with uint32_t
each; not sure what would be more sensible.  This of course
requires an API bump, although it should be backwards
compatible.

Any feedback on this is highly appreciated.


Cheers,
Franco