On 27 November 2014 at 09:25, Hans Petter Selasky <hps_at_selasky.org> wrote: > On 11/27/14 18:20, Adrian Chadd wrote: >> >> On 27 November 2014 at 09:18, Adrian Chadd <adrian_at_freebsd.org> wrote: >>> >>> Erk - SCTP folk - are you using the mbuf flowid field for something >>> SCTP specific? >> >> >> erk - yes, you are. >> >> It seems we're going to run into "what exactly should flowid be used >> for" problems. >> > > Hi, > > If the flowid has special meaning inside the SCTP, please define an own > "rsstype" for it. As far as I could see, it is only used to spread the > traffic on the network adapter and on the CPU cores. I'm more worried about at what layers we're going to be using the flowid values and that various pieces may stomp over other pieces. With the RSS stuff enabled, the IPv4 (and soon IPv6) input paths will re-hash an input frame if it doesn't have an RSS hash; it'll then overwrite whatever flowid value is in the mbuf. This is so no matter what the NIC hands us or what de-encapsulation hands us, we will always put that flow on into the right RSS bucket and a consistent CPU. This should mostly alleviate any out-of-order issues seen with internet-facing machines where things handle fragments and such. Then for the output side of things, it'll have to do a software RSS hash on frames that don't have an RSS hash. Right now I assume in the TCP path that the inp has an RSS flow setup in the inp and that the TCP timers and other assorted stuff uses the inp details. For the UDP output side of things I currently always re-calculate the RSS hash and stamp the mbuf with it before we send it out, again so it ends up on the same output RSS bucket and thus CPU / NIC queue. If the inp ends up with a different flowid (eg a hardware ring flowid) then: * it won't match up on the receive side, so the whole RSS lock contention avoidance thing can't happen; * there's a known mapping for RSS bucket -> CPU IDs (since we're not doing RSS bucket -> CPU ID reassignment yet) - but not for other flowid types to CPU IDs. In general I think the tidyup patch looks fine and I'll do some more RSS testing once it's committed. (But you did sneak in your new hash type in the diff :-) I think we then need to actually plan out how this stuff should hold together before we all rush into it or we'll end up with a mess of components that can't actually interact together. I don't want to have to explain to people that they can't use SCTP, RSS and hardware ring / flow assignment at the same time. It's just going to be painful in the long run. -adrianReceived on Thu Nov 27 2014 - 16:34:00 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:54 UTC