Re: More ULE bugs fixed.

On Wed, 15 Oct 2003, Daniel Eischen wrote:

> On Wed, 15 Oct 2003, Jeff Roberson wrote:
> 
> > I fixed two bugs that were exposed due to more of the kernel running
> > outside of Giant.  ULE had some issues with priority propagation that
> > stopped it from working very well.
> > 
> > Things should be much improved.  Feedback, as always, is welcome.  I'd
> > like to look into making this the default scheduler for 5.2 if things
> > start looking up.  I hope that scares you all into using it more. :-)
> 
> Before you do that, can you look into changing the scheduler
> interfaces to address David Xu's concern with it being
> suboptimal for KSE processes?
> 
There is also some work that I'd like to get done re:
cleaning up the scheduler interface a bit..

I know that Jeff and I have doiscussed this before but it was a long
time ago, and I've forgotten a lot and also learned a bit since then..

Here's my logic on the matter:

Any process has a number (fixed or variable) of kernel entities tghat
can be scheduled. In KSE (gotta get a better name) there are a variable
number of them. In libthr they are 1:1.

I would postulate that the action of scheduling these items in a fair
way is up to the scheduler. I had a very crude fairness module
added to the BSD4.4 scheduler but I think that fairness
is a property of the scheduler and not of the threading package.

If the scheduler doesn't care if threads are scheduled fairly than it
can just schedule all threads equally. I would say that the ksegrp 
in question (which represents a rough unit of 'fairness'), should 
make a call to the scheduler on creation specifying the required
concurrancy.

At the moment KSE-M:N based ksegrps would specify N = NCPU, and
THR based ksegrps would specify N = NTHREADS.
KSE-1:1 runs with a KSEGRP with a concurrancy of 1 per thread.

(I still think that THR should allocate a KSEGRP per thread not a KSE
but it's not critical.)

Basically What I'm saying is that each scheduler should taka
a concurrency setting for each KSEGRP and how it implements it
is hidden from higher layers.  The current 4.4 scheduler would 
implement it using KSEs and the existitng code but other shcedulers may
chose to implement it in different manners.

I think the top layer API calls for the scheduler should be:
setrunnable(thread) 
choosethread()
sched_clocktick()
sched_set_concurrancy()
(plus all the other 'entrypoints')


I think that the scheduler needs to be in control of scheduling threads 
because there is too much inside information needed for it to be done 
properly by an outside entity. For example if the scheduler is not a 
priority based scheduler then an outside entity can not know 
how to juggle which thread should be run next if there is a choice of
which to do..

this would mean that each scheduler would neeed its own module to 
do this juggling instead of having a separate module to do it..

it makes the job of the scheduler more difficult, but in fact it has to
be so, because true posix process-scope threads require that the
scheduler do this work.


a thread is made runnable (with a unix priority)
the scheduler needs to look at this thread in the context of all the
other threads from this process, the current concurrency rule for that
ksegrp and the other runnable threads, and adjust things so that:
1/ the new thread is run some time
2/ the ksegrp doesn't get TOO MUCH cpu, possibly
punishing other threads in the group to compensate..

This is all up for discussion, but it's my current thinking.

Julian