sio vs sched_lock LOR (was: Re: kern/68442: panic - acquiring duplicate lock of same type: "sleepq chain")

At 08:39 29/06/2004, Daniel Lang wrote:
>login: lock order reversal
> 1st 0xc070a0c0 sched lock (sched lock) _at_ /usr/src/sys/kern/kern_proc.c:672
> 2nd 0xc0745d40 sio (sio) _at_ /usr/src/sys/dev/sio/sio.c:3205
>Stack backtrace:
>backtrace(ffffffff,c0712948,c0712b00,c06e25c4,c07358dc) at 0xc051e066 = backtra2
>witness_checkorder(c0745d40,9,c06b1e34,c85,3f8) at 0xc05393c4 = witness_checkor4
>_mtx_lock_spin_flags(c0745d40,0,c06b1e34,c85,c0712498) at 0xc0516e9e = _mtx_loce
>siocnputc(c06f9c40,63) at 0xc064375f = siocnputc+0x6b
>cnputc(63) at 0xc05483ed = cnputc+0x4d
>putchar(63,e5bd87e0) at 0xc053387a = putchar+0x92
>kvprintf(c069d236,c05337e8,e5bd87e0,a,e5bd8800) at 0xc0533a87 = kvprintf+0x77
>printf(c069d236,32cf1,0,32ce6,0,dd8,c9115828) at 0xc0533763 = printf+0x43
>calcru(c91156e0,e5bd8ad0,e5bd8ad8,0,e5bd8a10) at 0xc051cb9e = calcru+0x1f2
>[...]

Regardless of the questionable validity of the other problems this machine
is encountering, this LOR is real (and ugly).  calcru() asserts that it is
holding sched_lock, but it can printf() about negative runtime or runtime
going backwards.  With a serial console attached, these messages end up at
siocnputc(), which needs sio_lock.

Now, I presume that there is some reason why the locking order hard-coded
into witness makes it illegal to pick up sio_lock while holding sched_lock,
but I can't see it.  Maybe someone else can explain?

In the mean time, a workaround for this LOR is to unplug the serial console. :-)

Colin Percival