Re: Increasing MAXPHYS

In message: <D9D66012-16FD-4FB6-AB6A-9A8D17727901_at_samsco.org>
            Scott Long <scottl_at_samsco.org> writes:
: I'd like to go in the opposite direction.  The queue-dispatch-queue
: model of GEOM is elegant and easy to extend, but very wasteful for
: the simple case, where the simple case is one or two simple
: partition transforms (mbr, bsdlabel) and/or a simple stripe/mirror
: transform.  None of these need a dedicated dispatch context in order
: to operate.  What I'd like to explore is compiling the GEOM stack at
: creation time into a linear array of operations that happen without
: a g_down/g_up context switch.  As providers and consumers taste each
: other and build a stack, that stack gets compiled into a graph, and
: that graph gets executed directly from the calling context, both
: from the dev_strategy() side on the top and the bio_done() on the
: bottom.  GEOM classes that need a detached context can mark
: themselves as such, doing so will prevent a graph from being
: created, and the current dispatch model will be retained.

I have a few things to say on this.

First, I've done similar things at past companies for systems that are
similar to geom's queueing environment.  It is possible to convert the
queueing nodes in the graph to filtering nodes in the graph.  Another
way to look at this is to say you're implementing direct dispatch into
geom's stack.  This can be both good and bad, but should reduce
latency a lot.

One problem that I see is that you are calling into the driver from a
different set of contexts.  The queueing stuff was there to protect
the driver from LoRs due to its routines being called from many
different contexts, sometimes with other locks held (fact of life
often in the kernel).

So this certainly is something worth exploring, especially if we have
optimized paths for up/down for certain geom classes while still
allowing the current robust, but slow, paths for the more complicated
nodes in the tree.  It remains to be see if there's going to be issues
around locking order, but we've hit that with both geom and ifnet in
the past, so caution (eg, running with WITNESS turned on early and
often) is advised.

Warner