Re: [rfc] bind per-cpu timeout threads to each CPU

On 19.02.2014 22:04, Adrian Chadd wrote:
> On 19 February 2014 11:59, Alexander Motin <mav_at_freebsd.org> wrote:
>
>>> So if we're moving towards supporting (among others) a pcbgroup / RSS
>>> hash style work load distribution across CPUs to minimise
>>> per-connection lock contention, we really don't want the scheduler to
>>> decide it can schedule things on other CPUs under enough pressure.
>>> That'll just make things worse.
>
>> True, though it is also not obvious that putting second thread on CPU run
>> queue is better then executing it right now on another core.
>
> Well, it depends if you're trying to optimise for "run all runnable
> tasks as quickly as possible" or "run all runnable tasks in contexts
> that minimise lock contention."
>
> The former sounds great as long as there's no real lock contention
> going on. But as you add more chances for contention (something like
> "100,000 concurrent TCP flows") then you may end up having your TCP
> timer firing stuff interfere with more TXing or RXing on the same
> connection.

100K TCP flows probably means 100K locks. That means that chance of lock 
collision on each of them is effectively zero. More realistic it could 
be to speak about cache coherency traffic, etc, but I still think that 
number of expired timeouts should be much lower then number of other 
flow data accesses.

> Chasing this stuff down is a pain, because it only really shows up
> when you're doing lots of concurrency.
>
> I'm happy to make this a boot-time option and leave it off for the
> time being. How's that?

I generally hate tunables like that. There are too few people who may 
even try to make grounded decision in that question. If you think it 
right -- just do it, otherwise -- don't do it. I am not really 
objecting, more like sounding concerns.

-- 
Alexander Motin