Re: SMP and setrunnable()- scheduler 4bsd

On Tue, 22 Jul 2003, Andy Farkas wrote:

> On Wed, 9 Jul 2003, Terry Lambert wrote:
> > Andy Farkas wrote:
> > > On Tue, 8 Jul 2003, Julian Elischer wrote:
> > > > It looks tp me that if we make a thread runnable
> > > > and there is a processor in the idle loop, the idle processor should be
> > > > kicked in some way to make it go get the newly runnable thread.
> > >
> > > Is this what's happenning to me an my setiathomes?
> >
> > The command:
> >
> > 	sysctl machdep.cpu_idle_hlt
> >
> > will tell you.  If it says 1, then it may be the problem.  If it
> > says 0, then it's not the problem.  You can explicitly set it to
> > zero to disable halting in the idle loop.  If you do this, your
> > machine is likely to run ~20% hotter (depending on the CPU type),
> > since HLT'ing an idle CPU tends to cool it off.
> >
> > If this fixes your problem, then it's likely that what's happening
> > is that one or more of your CPU's are being idled until the clock
> > or some other interrupt fires, at which point in time your setiathome
> > processes are probably not the highest priority, as other things with
> > higher priority have gotten in the run queue ahead of them.
> >
> > If this is the case, then Julian's suggested fix of an IPI to one or
> > more idle CPU's when a process becomes ready-to-run will most likely
> > be necessary to avoid this situation.
> 
> Well, looks like machdep.cpu_idle_hlt affects the 4bsd scheduler.
> 
> Julian, have your patches at http://www.freebsd.org/~julian/it.diff been
> incorporated yet? SHould I try them?

Mr Wolf,

They are not incorporated..
you may try them (but I didn't see much difference)
I would like to readress the issue some time.


> 
> hummer# sysctl -a | grep kern.ver
> kern.version: FreeBSD 5.1-RELEASE #0: Thu Jun 26 01:57:42 EST 2003
> hummer# sysctl -a | grep cpu
> kern.threads.virtual_cpu: 4
> kern.ccpu: 1948                 <--- whats this?

no idea

> kern.smp.cpus: 4
> hw.ncpu: 4
> machdep.cpu_idle_hlt: 0
> machdep.hlt_cpus: 0
> hummer#
> hummer# vmstat -c 12 -w 10
>  procs      memory      page                    disks     faults      cpu
>  r b w     avm    fre  flt  re  pi  po  fr  sr ad0 am0   in   sy  cs us sy id
>  1 1 3  124804  20792   99   0   0   0 131  20   0   0  362    0 544 33  3 64
>  2 1 2  125240  32460  197   0   0   0 358 258   0   1  365    0 2161 67 26  7
>  2 1 2  124728  32204  214   0   0   0 249   0   0   1  382    0 1280 72 23  5
>  1 1 3  124112  29652  257   0   0   0 247   0   3   1  414    0 1571 70 25  5
>  3 1 1  124624  26020  236   0   0   0 297   0   0   1  373    0 1367 70 24  7
>  2 1 2  124792  25464  222   0   0   0 266   0   2   1  408    0 1308 72 24  4
>  3 1 1  123764  25172  203   0   0   0 227   0   0   1  380    0 1053 72 23  5
>  3 1 2  124164  22004  229   0   0   0 264   0   0   1  371    0 1164 72 25  3
>  3 1 2  124796  20232  208   0   0   0 262   0   1   1  383    0 1269 72 23  5
>  3 1 1  123768  20448  225   0   0   0 252   0   0   1  374    0 1306 72 25  3
>  3 1 0  122736  33736  214   0   0   0 458 194   0   1  363    0 1341 72 24  4
>  3 1 1  123380  32008  236   0   0   0 222   0   2   1  405    0 1430 72 25  4
> hummer#
> hummer# sysctl machdep.cpu_idle_hlt=1
> machdep.cpu_idle_hlt: 0 -> 1
> hummer# vmstat -c 12 -w 10
>  procs      memory      page                    disks     faults      cpu
>  r b w     avm    fre  flt  re  pi  po  fr  sr ad0 am0   in   sy  cs us sy id
>  2 1 1  124760  20848   99   0   0   0 131  20   0   0  362    0 544 33  3 64
>  1 1 3  123948  20844  157   0   0   0 191   0   0   1  385    0 1142 51 21 28
>  2 1 2  124408  20472  134   0   0   0 176   0   0   1  372    0 1118 48 22 30
>  2 1 1  124992  18800  179   0   0   0 181   0   1   2  394    0 1147 49 20 31
>  2 2 3  123792  31228  162   0   0   0 367 196   0   1  378    0 1111 48 21 30
>  1 1 4  124348  30048  175   0   0   0 169   0   0   1  368    0 1084 51 22 27
>  2 0 4  122228  29140  142   0   0   0 193   0   1   2  399    0 1056 53 21 26
>  2 1 2  124688  25324  176   0   0   0 135   0   0   1  370    0 1138 47 21 31
>  2 1 1  122632  26572  136   0   0   0 209   0   0   1  374    0 1110 50 21 30
>  2 1 2  122288  26036  162   0   0   0 193   0   1   2  384    0 1152 49 22 29
>  2 1 2  122288  25204  136   0   0   0 157   0   0   1  369    0 1101 50 21 29
>  2 1 2  122700  23720  172   0   0   0 183   0   0   1  376    0 1123 48 21 31
> 
> Currently (cpu_idle_hlt=1) the load is fluctuating between 2.20 and 3.60
> every few minutes! (xload looks like a graph of a sinewave)
> 
> If I set cpu_idle_hlt back to 0 the load goes back to a steady 3.80 where
> it should be.

define "should".




> 
> --
> 
>  :{ andyf_at_speednet.com.au
> 
>         Andy Farkas
>     System Administrator
>    Speednet Communications
>  http://www.speednet.com.au/
> 
> 
> 
> 
>