Re: [rfc] bind per-cpu timeout threads to each CPU

On Wednesday, February 19, 2014 3:04:51 pm Adrian Chadd wrote:
> On 19 February 2014 11:59, Alexander Motin <mav_at_freebsd.org> wrote:
> 
> >> So if we're moving towards supporting (among others) a pcbgroup / RSS
> >> hash style work load distribution across CPUs to minimise
> >> per-connection lock contention, we really don't want the scheduler to
> >> decide it can schedule things on other CPUs under enough pressure.
> >> That'll just make things worse.
> 
> > True, though it is also not obvious that putting second thread on CPU run
> > queue is better then executing it right now on another core.
> 
> Well, it depends if you're trying to optimise for "run all runnable
> tasks as quickly as possible" or "run all runnable tasks in contexts
> that minimise lock contention."
> 
> The former sounds great as long as there's no real lock contention
> going on. But as you add more chances for contention (something like
> "100,000 concurrent TCP flows") then you may end up having your TCP
> timer firing stuff interfere with more TXing or RXing on the same
> connection.
> 
> Chasing this stuff down is a pain, because it only really shows up
> when you're doing lots of concurrency.
> 
> I'm happy to make this a boot-time option and leave it off for the
> time being. How's that?

I think having it be a tunable would be good.  OTOH, I could also
see another option which would be to pin all clock threads except
for the "default" one by default and only have the option control
whether or not the default thread is pinned to CPU 0 as callers
who use callout_on() are explicitly asking to run the callout on a
specific CPU.

-- 
John Baldwin