Re: Best thread pool implementation for the base system?

On Thu, Oct 24, 2019 at 12:57 PM Ian Lepore <ian_at_freebsd.org> wrote:

> On Thu, 2019-10-24 at 12:46 -0600, Alan Somers wrote:
> > I count 5 thread pool implementations in contrib:
> > * cddl/compat/opensolaris/misc/thread_pool.c
> > * contrib/apr-util/misc/apr_thread_pool.c
> > * contrib/llvm/lib/Support/ThreadPool.cpp
> > * contrib/openmp/runtime/src/kmp_tasking.cpp
> > * contrib/ofed/opensm/complib/cl_threadpool.c
> >
> > However, I can't find any examples outside of contrib.  I'd like to
> > use one
> > in /sbin/geli.  Shall I roll my own (as everybody else apparently
> > does), or
> > is there something I'm not aware of?
> >
> >
>
> Whenever the subject of thread pools comes up at $work I pose this
> question:  What task is it you need to accomplish where the cost of
> pthread_create/delete is significant in relation to the actual work
> that will be done during the lifetime of the thread?
>
> Over the years, the answers have been such that we never created a
> thread pool class or helpers.  What we did eventually come up with was
> basically an async taskqueue that had a single thread, because almost
> always the answer to the question was something like "the work to be
> done is really tiny and not-time-critical, it just needs to be done on
> a different thread to avoid [recursion|deadlock|whatever]."
>
> So I'd say the first thing to do is be sure that the best solution
> isn't just to pthread_create() as needed.  If it turns out the cost of
> pthread_create() is like 1-2% of the total work to be done, you may not
> need a pool of pre-created threads.
>
> -- Ian
>

Well, the time needed by pthread_create/delete is probably not
significant.  But the memory consumption is.  I could have up to 400 tasks,
and I don't want to run all of them in parallel because all of those stacks
add up.  But I definitely need some amount of parallelism.

FYI my interest is in speeding up "geli attach" by parallelizing the I/O to
multiple disks.  Currently it attaches to the disks one at a time.

-Alan