Re: cvs commit: src/sys/kern sched_ule.c

On Mon, 13 Dec 2004, Julian Elischer wrote:

>
>
> Jeff Roberson wrote:
>
> >On Mon, 13 Dec 2004, Julian Elischer wrote:
> >
> >
> >
> >>The whole problem that "slots" is trying to solve is to stop a single
> >>process
> >>from being able to flood the system with threads and therefore make the
> >>system
> >>unfair in its favour.
> >>
> >>The "slots" method is really suitable for the 4bsd scheduler but it is
> >>really
> >>not so good for ULE (at least I think that there are probably better
> >>ways that
> >>ULE could implement fairness).
> >>
> >>What I think should happen at this stage is that the inclusion of
> >>kern_switch.c
> >>should be replaced by actually copying the contents of that file into
> >>the two
> >>schedulers and that they be permitted to diverge. This would allow ULE and
> >>BSD to be cleaned up in terms of the sched_td/kse hack (where they are in
> >>fact the same structure, but to keep diffs to a minimum I defined one in
> >>terms of the other with macros).
> >>
> >>It would also allow jeff to experiment absolutly freely on how ULE might
> >>implement fairness without any constraints of worrying about the BSD
> >>scheduler, and visa versa.
> >>
> >>I have been hesitant to do this because there was some (small) amount of
> >>work going on in the shared file, but I think it is time to cut the
> >>umbilical
> >>cord. If ULE is really fixed then this would be a good time to break
> >>them apart,
> >>and delete kern_switch.c (or at least move most of the stuff in it out
> >>to the
> >>two schedulers). This would protect ULE from future problems being
> >>"imported" from BSD for example.
> >>
> >>comments?
> >>
> >>
> >
> >Why don't we move the ke_procq into the thread and then kern_switch can
> >remain with the generic runq code?  Then we can move *runqueue into the
> >individual schedulers.  At least then we won't have to make a copy of the
> >bit twiddling code.
> >
>
> hmm just noticed that both 4bsd and ule have kse structure (td_sched) fields
> that are not used any more. (e.g. ke_kglist, ke_kgrlist)
>
> The bit twiddling code is already separate in runq.c is it not?

No, it's in kern_switch.c too.

> The fact that finctions in kern_switch are currently used by both BSD
> and ULE
> doesn'rt make them "generic" from my perspective. The are just shared for
> historical reasons. runq_remove and runq_add (for example) ar epretty
> generic but would still need changing if a thread were on >1 list.
>
> setrunqueue() and remrunqueue() are heavily based on what fairness
> method is used. I'm not happy with the SLOTS as the ultimate answer,
> only as the easiest (except for "ignore fairness"). Having them
> generic limits wow this might be changed. Certainly if ULE were to implement
> a smarter fairness method it's need to have its own copy of them.

I see no reason to do anything other than a count of the number of threads
which are allowed to run.  This is what I originally suggested when I was
saying we didn't need a struct kse.

>
> Re. moving ke_procq. (should be renamed to ke_runq_entry or somrthing)
> What if a scheduler wants to keep a thread on TWO lists..
> Puting it int he scheduler independent part of teh thread structure
> makes this
> harder to do.
>
> For example, I would like to experiment with a version of the BSD
> scheduler that keeps a
> thread on BOTH teh percpu queue and an independent queue.
> it gets removed from both when selected, but suelction is done from teh
> pcpu queue
> first, and proceeds to teh general queue only if there is nothing for
> that cpu.
>
> Another example would be a scheduler that uses (I forget the propper name)
> probablity scheduling  rather than run queues. it would require a
> completely differnt set of
> fields to represent it's internal structures. Having an externally
> visible run queue
> would be misleading because it would be visible but not used.
>
> I would actually go the other way..
> td_runq should be moved to the td_thread as it is used by the fairness
> code only and that could be implemented completely differently by
> different schedulers. I left it where it was only for diff reduction
> reasons.
>
> Is there a real reason that the two scheduelrs should not have separate
> copies of this code other than diskspace? I think that maintainance
> might even be
> made easier if people maintaining them don't have to always bare in mind
> the fact that the code is being used in two different scenareos with
> very different
> frameworks around them.

It's hard for me to argue against endless hypothetical system arrangements
which may be made if we don't make any code generic at all.  I don't feel
that the potential introduction of these systems warrents copy and pasting
huge volumes of code into each scheduler.  We don't have this kind of
flexibility elsewhere in the system, and one might argue with good reason.
In fact, if you look at the general trend in FreeBSD, it is towards more
static systems that are more robust and reliable.  This is the case with
vnodes, where people are favoring moving away from the dynamic loadable
VOPs, and instead towards static type-checked functions.

Regardless, it's easy to make a case against copy & paste and code
duplication.  I think generally people would agree that duplicated code is
a bad thing, and not an aid to maintainability.  We need to impose some
structure on the scheduling interface or it will become too bloated and
too loosely coupled for anyone to reasonably understand.  It's already
well on its way there now.  If you paste kern_switch.c into sched_4bsd.c
its line count will go from 1,216 to 2225.  That's over an 80% increase.

Instead, why don't we simply move the slot code into the generic struct
ksegrp and proc.h.  With a few other minor changes, we could make
kern_switch.c compile on its own again, and be a real independent file,
rather than included directly in other source files.  We will then have no
mention of 'kse' outside of sched_* as you originally wanted.  I can then
go about making sched_ule.c sane again and remove the mess of defines and
flag respositioning that has resulted from a constantly unstable process
substructure arrangement.

If you really have to have the flexibility to do some other method of
concurrency, I suggest we make a table of operations for a process to call
to schedule a thread.  That would mean calling setrunqueue(),
choosethread(), adjustrunqueue(), and maybe_preempt() through indirect
pointers.  This would mean we could use much simpler versions of these
functions for single threaded programs as well, which would likely reduce
cpu overhead.


>
>
>
> julian
>