Re: Comments on the KSE option

I think I accidently deleted a line in my final note.. rewriting it below...


Julian Elischer wrote:
> John, I appreciate that you have made KSE an option, but the way you 
> have done it shows a complete misundertanding of what is there.
> 
> What you are calling "KSE" is in fact several different facilities that
> are orthogonal.  The one that you have the most trouble with is in fact 
> not SA based threading (refered to by most people as "KSE" but, rather
> the fair scheduling code).
> 
> The aim of the fair scheduling code is to ensure that if you, as a user, 
> make a process that starts 1000 threads, and I as a user, make an 
> unthreaded process, then I can still get to the CPU at somewhat similar
> rates to you.  A naive scheduler would give you 1000 cpu slots and me 1.
> 
> the current fair scheduler tries to make sure that each process gets
> a fair crack at the CPU by holding back some of the runnable threads 
> from the threadded process, until the ones it has in therun queu have 
> been completed.. A bit like telling a young child, "yes you can have 
> more ice-cream, when you've finished the ice-cream you already have".
> 
> I note that David recently (in the last year) disabled the fair
> scheduling capacity of the libthr  code, but he didn't do it quite right
> so that it still does all the work for it, and then disregarded the 
> result. This means that not only does a 1000 thread process (libthr)
> completely push a nonthreaded process out of the system, but it pays
> all the costs in the scheduler for working out how to NOT do that.
> 
> 
> The fairness algorythm that you have made 'optional' is a very crude one 
> and I had thought that by now someone would have written a better one,
> but no-one has.
> 
> I suggest that you fix your patch in this way:
> you need (at least) 2 options.
>     KSE
> and
>     FAIR_THREADS
> 
> most of the improvements you are seeing comes from the second one.
> Especially all your changes that are in the scheduler. This removes the 
> fair scheduling capability. It affects all threading libraries that
> do not deliberatly knacker it. In other words it should be orthogonal
> to what threading library is running.
> 
> If it is made a project goal that threads should be unfair, then
> I have no objections to removing the code, but it needs to be a decision
> that is deliberately taken. It was an initial project goal that threads 
> should be fair, and the fact that David has made it ineffective for
> libthr (though he still pays the full price for it) is not a reason to
> throw it out. (What he does is to assign a new KSEGRP for each thread, 
> but he doesn't label it  as exempt from fairness so it does all the
> work only to discover at the end that it is the only thread on the 
> ksegrp, and therefore always eligible to run).
> If the correct flags were set, then then David's threads
> could probably get the same speedup as seen with the KSE option removed,
> as all the overhead would be skipped, but then we would be officially
> condoning unfair threading.
> teh chage to do thos would be to add a ksegrp or thread flag (possibly 
> thread) called TDF_FAIR_SCHED
> 
> and change the few lines in the scheduler that do:
>         if ((td->td_proc->p_flag & P_HADTHREADS) == 0) {
> 
> to be
>     if ((td->flags & TDF_FAIR_SCHED) == 0) {
> 
> 
> and set that flag in the  threading libraries when threads should be
> made fair.  then probably the entire advantage seen by David in the 
> supersmack tests from unsetting KSE would be seen by simply not setting
> that bit.
> 
> (it might also just look for:
>       if (td->ksegrp->kg_numthreads == 1)
> and achieve the same thing automatically.
> 
> 
> So, the question is:
> DO we as a project want to have fair threading or unfair threading?
> 
> Should processes with a lot of threads be able to push out processes 
> that do the same thing by using a state machine or an event loop?
> 
> BTW another alternative would be to write a different scheduler,
> called sched_4bsd-unfair  (or similar) and just strip out the fairness 
> code.  it would be another way of doing much the same thing.
> 
> This is a completely different question to whether there should be
> an M:N threading library, the existance of which should make no
> noticable difference to the speed of processses that don't use it.
> 
> My moral for this story is.
> "If you don't understand the bigger picture and you modify things
> then you can expect that your modifications may have unforseen
> circumstances."
> 
> I as well as most other people fall foul of this at various times in our
> carreers.
> 
> 
> 
> ============
> Technical note:
> 

> The current fairness code relies on a sub structure of the proc,
> called a ksegrp. This structure represents the "unit of fairness".
> Most processes have one of these so they act as if the unit of
> fairness is the entire process. The concept was that a threaded
> process would have one of these for it's directly allocated threads,
> and that they would act, as a group, fairly towards the rest of the
> system. A process could also have a library that unbeknownst to the
> program propper, would create its own ksegrp, with its own threads,
> that would act independently and have their own 'fairness'
> characteristics, priorities etc.


In a fair threaded process with may runnable threads,

> only the top N (= ncpu usually) threads are allowed onto the system
> run queue to compete with other processes. 

In libthr,

 > By assigning a separate
> KSEGRP for each thread the code assures that each thread is
> immediatly promoted to the system run queue, however because the
> system code doesn't realise that he is trying to subvert the fairness
> code, it still takes the code path that looks at the ksegrp run queues
> and does all sorts of other checks.
> 
> If someone can come up with a better fairness method (Please!) then 
> I'm happy to see all that code in the shceduler replaced by whatever 
> else is chosen (nothing if we REALLY want to see thread unfairness).
> 
> I think that libthr should be moved back to be "fair" by default, and
> that unfair mode should be made optional (if you are root) so that
> dedicated servers, where the administrator wants to get all the 
> performance, and is willing to state explicitly that fairness is not
> important to him, can do just that (and for benchmarks).
> 
> 

> 
> 
> 
> 
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"