Re: [rfc] bind per-cpu timeout threads to each CPU

On Wednesday, February 19, 2014 4:02:54 pm John Baldwin wrote:
> On Wednesday, February 19, 2014 3:04:51 pm Adrian Chadd wrote:
> > On 19 February 2014 11:59, Alexander Motin <mav_at_freebsd.org> wrote:
> > 
> > >> So if we're moving towards supporting (among others) a pcbgroup / RSS
> > >> hash style work load distribution across CPUs to minimise
> > >> per-connection lock contention, we really don't want the scheduler to
> > >> decide it can schedule things on other CPUs under enough pressure.
> > >> That'll just make things worse.
> > 
> > > True, though it is also not obvious that putting second thread on CPU run
> > > queue is better then executing it right now on another core.
> > 
> > Well, it depends if you're trying to optimise for "run all runnable
> > tasks as quickly as possible" or "run all runnable tasks in contexts
> > that minimise lock contention."
> > 
> > The former sounds great as long as there's no real lock contention
> > going on. But as you add more chances for contention (something like
> > "100,000 concurrent TCP flows") then you may end up having your TCP
> > timer firing stuff interfere with more TXing or RXing on the same
> > connection.
> > 
> > Chasing this stuff down is a pain, because it only really shows up
> > when you're doing lots of concurrency.
> > 
> > I'm happy to make this a boot-time option and leave it off for the
> > time being. How's that?
> 
> I think having it be a tunable would be good.  OTOH, I could also
> see another option which would be to pin all clock threads except
> for the "default" one by default and only have the option control
> whether or not the default thread is pinned to CPU 0 as callers
> who use callout_on() are explicitly asking to run the callout on a
> specific CPU.

(A further variant of this would be to divorce cpu0's swi from the
catch-all softclock and let the catch-all softclock float, but bind
all the per-cpu swis)

-- 
John Baldwin