Re: SMP and setrunnable()- scheduler 4bsd

From: Andy Farkas <andyf_at_speednet.com.au>
Date: Tue, 22 Jul 2003 09:27:14 +1000 (EST)
On Wed, 9 Jul 2003, Terry Lambert wrote:
> Andy Farkas wrote:
> > On Tue, 8 Jul 2003, Julian Elischer wrote:
> > > It looks tp me that if we make a thread runnable
> > > and there is a processor in the idle loop, the idle processor should be
> > > kicked in some way to make it go get the newly runnable thread.
> >
> > Is this what's happenning to me an my setiathomes?
>
> The command:
>
> 	sysctl machdep.cpu_idle_hlt
>
> will tell you.  If it says 1, then it may be the problem.  If it
> says 0, then it's not the problem.  You can explicitly set it to
> zero to disable halting in the idle loop.  If you do this, your
> machine is likely to run ~20% hotter (depending on the CPU type),
> since HLT'ing an idle CPU tends to cool it off.
>
> If this fixes your problem, then it's likely that what's happening
> is that one or more of your CPU's are being idled until the clock
> or some other interrupt fires, at which point in time your setiathome
> processes are probably not the highest priority, as other things with
> higher priority have gotten in the run queue ahead of them.
>
> If this is the case, then Julian's suggested fix of an IPI to one or
> more idle CPU's when a process becomes ready-to-run will most likely
> be necessary to avoid this situation.

Well, looks like machdep.cpu_idle_hlt affects the 4bsd scheduler.

Julian, have your patches at http://www.freebsd.org/~julian/it.diff been
incorporated yet? SHould I try them?

hummer# sysctl -a | grep kern.ver
kern.version: FreeBSD 5.1-RELEASE #0: Thu Jun 26 01:57:42 EST 2003
hummer# sysctl -a | grep cpu
kern.threads.virtual_cpu: 4
kern.ccpu: 1948                 <--- whats this?
kern.smp.cpus: 4
hw.ncpu: 4
machdep.cpu_idle_hlt: 0
machdep.hlt_cpus: 0
hummer#
hummer# vmstat -c 12 -w 10
 procs      memory      page                    disks     faults      cpu
 r b w     avm    fre  flt  re  pi  po  fr  sr ad0 am0   in   sy  cs us sy id
 1 1 3  124804  20792   99   0   0   0 131  20   0   0  362    0 544 33  3 64
 2 1 2  125240  32460  197   0   0   0 358 258   0   1  365    0 2161 67 26  7
 2 1 2  124728  32204  214   0   0   0 249   0   0   1  382    0 1280 72 23  5
 1 1 3  124112  29652  257   0   0   0 247   0   3   1  414    0 1571 70 25  5
 3 1 1  124624  26020  236   0   0   0 297   0   0   1  373    0 1367 70 24  7
 2 1 2  124792  25464  222   0   0   0 266   0   2   1  408    0 1308 72 24  4
 3 1 1  123764  25172  203   0   0   0 227   0   0   1  380    0 1053 72 23  5
 3 1 2  124164  22004  229   0   0   0 264   0   0   1  371    0 1164 72 25  3
 3 1 2  124796  20232  208   0   0   0 262   0   1   1  383    0 1269 72 23  5
 3 1 1  123768  20448  225   0   0   0 252   0   0   1  374    0 1306 72 25  3
 3 1 0  122736  33736  214   0   0   0 458 194   0   1  363    0 1341 72 24  4
 3 1 1  123380  32008  236   0   0   0 222   0   2   1  405    0 1430 72 25  4
hummer#
hummer# sysctl machdep.cpu_idle_hlt=1
machdep.cpu_idle_hlt: 0 -> 1
hummer# vmstat -c 12 -w 10
 procs      memory      page                    disks     faults      cpu
 r b w     avm    fre  flt  re  pi  po  fr  sr ad0 am0   in   sy  cs us sy id
 2 1 1  124760  20848   99   0   0   0 131  20   0   0  362    0 544 33  3 64
 1 1 3  123948  20844  157   0   0   0 191   0   0   1  385    0 1142 51 21 28
 2 1 2  124408  20472  134   0   0   0 176   0   0   1  372    0 1118 48 22 30
 2 1 1  124992  18800  179   0   0   0 181   0   1   2  394    0 1147 49 20 31
 2 2 3  123792  31228  162   0   0   0 367 196   0   1  378    0 1111 48 21 30
 1 1 4  124348  30048  175   0   0   0 169   0   0   1  368    0 1084 51 22 27
 2 0 4  122228  29140  142   0   0   0 193   0   1   2  399    0 1056 53 21 26
 2 1 2  124688  25324  176   0   0   0 135   0   0   1  370    0 1138 47 21 31
 2 1 1  122632  26572  136   0   0   0 209   0   0   1  374    0 1110 50 21 30
 2 1 2  122288  26036  162   0   0   0 193   0   1   2  384    0 1152 49 22 29
 2 1 2  122288  25204  136   0   0   0 157   0   0   1  369    0 1101 50 21 29
 2 1 2  122700  23720  172   0   0   0 183   0   0   1  376    0 1123 48 21 31

Currently (cpu_idle_hlt=1) the load is fluctuating between 2.20 and 3.60
every few minutes! (xload looks like a graph of a sinewave)

If I set cpu_idle_hlt back to 0 the load goes back to a steady 3.80 where
it should be.

--

 :{ andyf_at_speednet.com.au

        Andy Farkas
    System Administrator
   Speednet Communications
 http://www.speednet.com.au/
Received on Mon Jul 21 2003 - 14:27:27 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:37:16 UTC