HEADSUP: Native preemption added to the kernel scheduler

In theory this is a big NOP except for some small optimizations in the form of 
avoiding a few context switches and avoiding some run queue operations.  
Several people have tested this code but there may be some remaining 
adventures.  Note that this adds a printf during dmesg for architectures that 
do not support preemption about preemption being disabled and degrading 
performance (mostly via increased latency).  Preemption is enabled by 
defining PREEMPTION in <machine/param.h> and architecture porters are 
encouraged to get preemption working on their architecture.

On Friday 02 July 2004 04:21 pm, John Baldwin wrote:
> jhb         2004-07-02 20:21:44 UTC
>
>   FreeBSD src repository
>
>   Modified files:
>     sys/alpha/alpha      interrupt.c
>     sys/alpha/include    param.h
>     sys/amd64/amd64      intr_machdep.c
>     sys/amd64/include    param.h
>     sys/conf             NOTES options
>     sys/i386/i386        intr_machdep.c
>     sys/i386/include     param.h
>     sys/ia64/ia64        interrupt.c
>     sys/kern             kern_intr.c kern_mutex.c kern_shutdown.c
>                          kern_switch.c kern_synch.c sched_4bsd.c
>                          sched_ule.c
>     sys/powerpc/powerpc  intr_machdep.c
>     sys/sparc64/sparc64  intr_machdep.c
>     sys/sys              interrupt.h proc.h
>     sys/vm               vm_zeroidle.c
>   Log:
>   Implement preemption of kernel threads natively in the scheduler rather
>   than as one-off hacks in various other parts of the kernel:
>   - Add a function maybe_preempt() that is called from sched_add() to
>     determine if a thread about to be added to a run queue should be
>     preempted to directly.  If it is not safe to preempt or if the new
>     thread does not have a high enough priority, then the function returns
>     false and sched_add() adds the thread to the run queue.  If the thread
>     should be preempted to but the current thread is in a nested critical
>     section, then the flag TDF_OWEPREEMPT is set and the thread is added
>     to the run queue.  Otherwise, mi_switch() is called immediately and the
>     thread is never added to the run queue since it is switch to directly.
>     When exiting an outermost critical section, if TDF_OWEPREEMPT is set,
>     then clear it and call mi_switch() to perform the deferred preemption.
>   - Remove explicit preemption from ithread_schedule() as calling
>     setrunqueue() now does all the correct work.  This also removes the
>     do_switch argument from ithread_schedule().
>   - Do not use the manual preemption code in mtx_unlock if the architecture
>     supports native preemption.
>   - Don't call mi_switch() in a loop during shutdown to give ithreads a
>     chance to run if the architecture supports native preemption since
>     the ithreads will just preempt DELAY().
>   - Don't call mi_switch() from the page zeroing idle thread for
>     architectures that support native preemption as it is unnecessary.
>   - Native preemption is enabled on the same archs that supported ithread
>     preemption, namely alpha, i386, and amd64.
>
>   This change should largely be a NOP for the default case as committed
>   except that we will do fewer context switches in a few cases and will
>   avoid the run queues completely when preempting.
>
>   Approved by:    scottl (with his re_at_ hat)
>
>   Revision  Changes    Path
>   1.79      +1 -1      src/sys/alpha/alpha/interrupt.c
>   1.34      +2 -0      src/sys/alpha/include/param.h
>   1.7       +1 -1      src/sys/amd64/amd64/intr_machdep.c
>   1.12      +2 -0      src/sys/amd64/include/param.h
>   1.1240    +6 -0      src/sys/conf/NOTES
>   1.459     +1 -0      src/sys/conf/options
>   1.7       +1 -1      src/sys/i386/i386/intr_machdep.c
>   1.71      +2 -0      src/sys/i386/include/param.h
>   1.46      +1 -1      src/sys/ia64/ia64/interrupt.c
>   1.111     +3 -16     src/sys/kern/kern_intr.c
>   1.140     +6 -0      src/sys/kern/kern_mutex.c
>   1.153     +24 -13    src/sys/kern/kern_shutdown.c
>   1.68      +93 -4     src/sys/kern/kern_switch.c
>   1.252     +4 -1      src/sys/kern/kern_synch.c
>   1.43      +11 -1     src/sys/kern/sched_4bsd.c
>   1.110     +10 -1     src/sys/kern/sched_ule.c
>   1.6       +1 -1      src/sys/powerpc/powerpc/intr_machdep.c
>   1.19      +0 -4      src/sys/sparc64/sparc64/intr_machdep.c
>   1.28      +1 -1      src/sys/sys/interrupt.h
>   1.384     +2 -0      src/sys/sys/proc.h
>   1.26      +2 -0      src/sys/vm/vm_zeroidle.c

-- 
John Baldwin <jhb_at_FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org