Re: More ULE bugs fixed.

Julian Elischer wrote:

>On Wed, 15 Oct 2003, Daniel Eischen wrote:
>
>  
>
>>On Wed, 15 Oct 2003, Jeff Roberson wrote:
>>
>>    
>>
>>>I fixed two bugs that were exposed due to more of the kernel running
>>>outside of Giant.  ULE had some issues with priority propagation that
>>>stopped it from working very well.
>>>
>>>Things should be much improved.  Feedback, as always, is welcome.  I'd
>>>like to look into making this the default scheduler for 5.2 if things
>>>start looking up.  I hope that scares you all into using it more. :-)
>>>      
>>>
>>Before you do that, can you look into changing the scheduler
>>interfaces to address David Xu's concern with it being
>>suboptimal for KSE processes?
>>
>>    
>>
>There is also some work that I'd like to get done re:
>cleaning up the scheduler interface a bit..
>
>I know that Jeff and I have doiscussed this before but it was a long
>time ago, and I've forgotten a lot and also learned a bit since then..
>
>Here's my logic on the matter:
>
>Any process has a number (fixed or variable) of kernel entities tghat
>can be scheduled. In KSE (gotta get a better name) there are a variable
>number of them. In libthr they are 1:1.
>
>I would postulate that the action of scheduling these items in a fair
>way is up to the scheduler. I had a very crude fairness module
>added to the BSD4.4 scheduler but I think that fairness
>is a property of the scheduler and not of the threading package.
>
>If the scheduler doesn't care if threads are scheduled fairly than it
>can just schedule all threads equally. I would say that the ksegrp 
>in question (which represents a rough unit of 'fairness'), should 
>make a call to the scheduler on creation specifying the required
>concurrancy.
>
>At the moment KSE-M:N based ksegrps would specify N = NCPU, and
>THR based ksegrps would specify N = NTHREADS.
>KSE-1:1 runs with a KSEGRP with a concurrancy of 1 per thread.
>
>(I still think that THR should allocate a KSEGRP per thread not a KSE
>but it's not critical.)
>
>Basically What I'm saying is that each scheduler should taka
>a concurrency setting for each KSEGRP and how it implements it
>is hidden from higher layers.  The current 4.4 scheduler would 
>implement it using KSEs and the existitng code but other shcedulers may
>chose to implement it in different manners.
>
>I think the top layer API calls for the scheduler should be:
>setrunnable(thread) 
>choosethread()
>sched_clocktick()
>sched_set_concurrancy()
>(plus all the other 'entrypoints')
>
>
>I think that the scheduler needs to be in control of scheduling threads 
>because there is too much inside information needed for it to be done 
>properly by an outside entity. For example if the scheduler is not a 
>priority based scheduler then an outside entity can not know 
>how to juggle which thread should be run next if there is a choice of
>which to do..
>
>this would mean that each scheduler would neeed its own module to 
>do this juggling instead of having a separate module to do it..
>
>it makes the job of the scheduler more difficult, but in fact it has to
>be so, because true posix process-scope threads require that the
>scheduler do this work.
>
>
>a thread is made runnable (with a unix priority)
>the scheduler needs to look at this thread in the context of all the
>other threads from this process, the current concurrency rule for that
>ksegrp and the other runnable threads, and adjust things so that:
>1/ the new thread is run some time
>2/ the ksegrp doesn't get TOO MUCH cpu, possibly
>punishing other threads in the group to compensate..
>
>This is all up for discussion, but it's my current thinking.
>  
>
These are what I want to see. Current I am forced to maintain nexus 
among kse and ksegrp and
thread whenever kse_create is called or thread is exiting but those 
nexuses is not used by
SA code at all.  From my view as a scheduler interface user,  I only 
want to see
ksegrp and thread,  the scheduler interface should be thread friendly 
not kse which is
scheduler internal detail to maintian scheduling faireness,  for 
example,  it might just
be a token to assign a  thread,  a thread has this token can be picked 
to run on physical CPU,
scheduler would  maintain token assigning faireness,  when there more 
tokens in ksegrp,
the more concurrent level the ksegrp will get.

Another problem is libkse in future would support cpu affinity, with 
current scheduler interface,
I can not specify a newly create thread to be bound to a specified CPU, 
this is needed by libkse,
when an upcall is scheduled, I want to specify which cpu is prefered to 
run newly created upcall
thread to give userland scheduler a stable cpu affinity state.
Dan ever asked to add kse_bind syscall to let him bind userland kse to a 
specified cpu,
if we want to support him, then we might need this feature in kernel.

>Julian
>  
>
David Xu