Re: Best thread pool implementation for the base system?

On Thu, 2019-10-24 at 12:46 -0600, Alan Somers wrote:
> I count 5 thread pool implementations in contrib:
> * cddl/compat/opensolaris/misc/thread_pool.c
> * contrib/apr-util/misc/apr_thread_pool.c
> * contrib/llvm/lib/Support/ThreadPool.cpp
> * contrib/openmp/runtime/src/kmp_tasking.cpp
> * contrib/ofed/opensm/complib/cl_threadpool.c
> 
> However, I can't find any examples outside of contrib.  I'd like to
> use one
> in /sbin/geli.  Shall I roll my own (as everybody else apparently
> does), or
> is there something I'm not aware of?
> 
> 

Whenever the subject of thread pools comes up at $work I pose this
question:  What task is it you need to accomplish where the cost of
pthread_create/delete is significant in relation to the actual work
that will be done during the lifetime of the thread?

Over the years, the answers have been such that we never created a
thread pool class or helpers.  What we did eventually come up with was
basically an async taskqueue that had a single thread, because almost
always the answer to the question was something like "the work to be
done is really tiny and not-time-critical, it just needs to be done on
a different thread to avoid [recursion|deadlock|whatever]."

So I'd say the first thing to do is be sure that the best solution
isn't just to pthread_create() as needed.  If it turns out the cost of
pthread_create() is like 1-2% of the total work to be done, you may not
need a pool of pre-created threads.

-- Ian