RE: Renice idle process to -20 causes hang.

On Wed, 23 Apr 2003, John Baldwin wrote:

> On 23-Apr-2003 Craig Reyenga wrote:
> > While running a samba benchmark, I tried 'renice -n -20 10' (pid 10 is idle
> > thread) to see what would happen. Turns out that this operation isn't
> > illegal, and the system hung accordingly. The benchmark timed out soon
> > after, and the console started displaying this message:
> >
> > swap_pager: indefinite wait buffer: device: ad0s1b, blkno: 776, size: 4096
> >
> > over and over, although not rapidly. I'm not sure what to blame; perhaps
> > setpriority() should return [EINVAL] or something. I can provide more info,
> > upon request.
>
> Umm, yeah, setpriority should do an EINVAL, but that's not probably the
> real bug.  idlethreads are never on the run queues, they are truly idle
> and only executed when there is nothing else to do.  They don't have a
> real priority other than "anything else is more important".  What might have
> happened is that setpiority() put the idle process on the run queue, which
> is guaranteed to totally hose your system.

There are also idprio threads (which should not be confused with truly
idle threads).  setpriority seems to change the nice value on all threads
except the swapper but this should have no other effect on threads with
non-default scheduling classes.  The scheduling classes and policies in
use are not easy to see.  To see some aspects of them, first fix ps so that
it doesn't follow a garbage pointer:

%%%
Index: print.c
===================================================================
RCS file: /home/ncvs/src/bin/ps/print.c,v
retrieving revision 1.82
diff -u -2 -r1.82 print.c
--- print.c	15 Apr 2003 18:49:20 -0000	1.82
+++ print.c	24 Apr 2003 10:53:52 -0000
_at__at_ -641,7 +649,7 _at__at_
 	char str[8];
 	unsigned class, level;
-
+
 	v = ve->var;
-	lpri = (struct priority *) ((char *)k + v->off);
+	lpri = &k->ki_p->ki_pri;
 	class = lpri->pri_class;
 	level = lpri->pri_level;
%%%

This patch also fixes a nearby style bug, but doesn't remove dead code in
keywords.c (v->off is correct, but there is no need for it since there is
only one priority field to print).

Some output from ps with the above patch:

%   UID   PID  PPID PRI NI  RTPRIO   VSZ  RSS STAT  TT       TIME COMMAND
%     0     0     0 -16  0  normal     0   12 DLs   ??    0:00.44 swapper
%     0     1     0   8 -1  normal   664  236 I<s   ??    0:00.68 init
%     0     2     0 -16  0  normal     0   12 DL    ??    0:00.57 pagedaemon

Many system processes have normal scheduling classes.  Some like init even
reach user mode, so niceness should affect them.

%     0     3     0 171  0 idle:25     0   12 DL    ??    0:17.00 pagezero

ps and the kernel need more patches to print the realtime priority.  25 here
is bogus; it is just priority level already printed in the PRI column, but
without a bias and messed up (the level is 255; subtracting PZERO gives 171,
and discarding the last digit gives 25).  The original realtime priority is
not available.  It can be recovered from pri_user for the relevant priority
classes, but this is too messy for applications to do so the kernel should
do it).

%     0     4     0 -16  0  normal     0   12 DL    ??    0:02.46 bufdaemon
%     0     5     0  -4  0  normal     0   12 DL    ??    0:02.09 vnlru
%     0     6     0  20  0  normal     0   12 DL    ??    0:34.08 syncer
%     0    10     0 -16  0  normal     0   12 DL    ??    0:00.00 ktrace
%     0    11     0 -16  0  normal     0   12 RL    ??  4159:04.53 idle

More kernel processes with a normal scheduling class.  "idle" at least
shouldn't be so normal.

%     0    12     0 -68  0    1:16     0   12 WL    ??    5:38.81 swi1: net
%     0    13     0 -68  0    1:16     0   12 LL    ??    6:41.09 swi6: tty:sio clock

ps needs more patches to print the ithread scheduling class.  "1:" here is
PRI_ITHD.

%     ...
%    15 13121 13116  76  0  normal 37984 24872 S     ??    1:00.33 netscape-bin
%    15 13122 13121  68  0  10:152 37984 24872 S     ??    0:04.28 netscape-bin

ps needs more patches to print the fifo scheduling policy.  "10:" here is
(PRI_REALTIME | PRI_FIFO).

%     ...
%     0 15072     1  44  0 real:12   664  396 Ss    ??    0:22.02 ntpd

Real-realtime-priority threads are mishandled in the same way as idle-
realtime-priority threads.  "12" here is (44 + PZERO) = 128 truncated.

Bruce