Re: Problem remains with FreeBSD 6.0-RC1 as seen in RELENG_5

From: Robert Watson <rwatson_at_FreeBSD.org>
Date: Wed, 19 Oct 2005 18:49:31 +0100 (BST)
On Wed, 19 Oct 2005, John Baldwin wrote:

> He'll need witness for 'show alllocks' but that will require having the 
> 'hptlock' spin mutex added to witness' spin lock order.  Also, the 
> psuedo-rw lock phk created for filedesc isn't witness checked since it 
> isn't a mutex or sx lock, and it maybe that there is a reversal 
> involving that lock, but it will be tricky to tell if that is the case.

His previous output includes "show witness", so I think he should be all 
set.

Robert N M Watson

>
>> Thanks,
>>
>> Robert N M Watson
>>
>>> Moving to the console, I get the same behaviour such that if it is
>>> already logged in, I can hit return in the shell and get a new shell
>>> prompt, but any command I try (i.e.  'uptime' or 'uname') then results in
>>> the same lack of any further response.  [Though, obviously from below,
>>> sending a break to activate DDB still works.]
>>>
>>> If the console is not already logged in and I hit return at "login:", I
>>> get a new "login:" prompt; but, as soon as I enter a username+^M or even
>>> Ctrl-D it, I cease to receive any feedback besides terminal echo from my
>>> input.
>>>
>>>
>>> This is:
>>> FreeBSD [host-elided] 6.0-RC1 FreeBSD 6.0-RC1 #0: Mon Oct 17 15:52:35 CDT
>>> 2005     root_at_[host-elided]:/usr/obj/usr/src/sys/SMP_DDB  i386
>>>
>>> # grep '^[a-z]' /sys/i386/conf/SMP_DDB
>>> include GENERIC
>>> ident           DDB
>>> options         SMP                     # Symmetric MultiProcessor Kernel
>>> options         INCLUDE_CONFIG_FILE     # Include this file in kernel
>>> options DDB                     #Enable the kernel debugger
>>> options DDB_NUMSYM              #Print numerical value of symbols too
>>> options KDB
>>> options KDB_TRACE
>>> options BREAK_TO_DEBUGGER
>>>
>>> [After notification of a problem, this is from the serial console...a few
>>> further inline comments included in brackets "[]".]
>>>
>>> #
>>> [Typed: Ctrl-T]
>>> load: 1.82  cmd: sh 795 [ttyin] 0.00u 0.00s 0% 168k
>>> # uptime
>>> [Waited a while to be sure it was hung and not returning. Retyped:
>>> Ctrl-T] load: 1.82  cmd: uptime 83303 [filelist lock] 0.00u 0.00s 0% 180k
>>> [Sent BREAK]
>>> KDB: enter: Line break on console
>>> [thread pid 82885 tid 100133 ]
>>> Stopped at	0xc0659b37 = kdb_enter+0x2b:	nop
>>> db> set $lines=0
>>> db> trace
>>> Tracing pid 82885 tid 100133 td 0xc3c34c00
>>> kdb_enter(c0895a5e) at 0xc0659b37 = kdb_enter+0x2b
>>> siointr1(c372ec00) at 0xc07f09b2 = siointr1+0xce
>>> siointr(c372ec00) at 0xc07f0792 = siointr+0x5e
>>> intr_execute_handlers(c3528090,f1325b68,4,f1325bcc,c0805073) at
>>> 0xc0808985 = intr_execute_handlers+0x85 lapic_handle_intr(34) at
>>> 0xc080b042 = lapic_handle_intr+0x2e
>>> Xapic_isr1() at 0xc0805073 = Xapic_isr1+0x33
>>> --- interrupt, eip = 0xc0685d59, esp = 0xf1325bac, ebp = 0xf1325bcc ---
>>> unp_gc(c43be2c8) at 0xc0685d59 = unp_gc+0xf1
>>> unp_detach(c43b7000,f1325c08,c067a188,c43be2c8,0) at 0xc06840fa =
>>> unp_detach+0xae uipc_detach(c43be2c8) at 0xc0682f8e = uipc_detach+0x82
>>> soclose(c43be2c8,c40fb6c0,0,f1325c34,c0620c10) at 0xc067a188 =
>>> soclose+0x278 soo_close(c40fb6c0,c3c34c00) at 0xc0669fb3 = soo_close+0x63
>>> fdrop_locked(c40fb6c0,c3c34c00,c3a3d800,f1325cb4,c061f0fb) at 0xc0620c10
>>> = fdrop_locked+0xac fdrop(c40fb6c0,c3c34c00,c066371f,c40fb6c0,c878ac00)
>>> at 0xc0620b5d = fdrop+0x41 closef(c40fb6c0,c3c34c00,0,c3c34c00,6) at
>>> 0xc061f0fb = closef+0x427 close(c3c34c00,f1325d04,1,0,292) at 0xc061c4bd
>>> = close+0x211
>>> syscall(3b,1003b,bfbf003b,80d58e7,80e8780) at 0xc0818557 = syscall+0x2bf
>>> Xint0x80_syscall() at 0xc0804d0f = Xint0x80_syscall+0x1f
>>> --- syscall (6, FreeBSD ELF32, close), eip = 0x282cb7d3, esp =
>>> 0xbfbfae4c, ebp = 0xbfbfae68 --- db> show reg
>>> cs		  0x20
>>> ds	    0xc0870028
>>> es	    0xc0650028	parse_dynamic+0x4
>>> fs	    0xf1320008
>>> ss		  0x28
>>> eax		  0x22
>>> ecx	    0xc1033000
>>> edx		     0
>>> ebx	    0xc3c34c00
>>> esp	    0xf1325afc
>>> ebp	    0xf1325afc
>>> esi	    0xc372ec00
>>> edi		  0xf9
>>> eip	    0xc0659b37	kdb_enter+0x2b
>>> efl		  0x86
>>> dr0		     0
>>> dr1		     0
>>> dr2		     0
>>> dr3		     0
>>> dr4	    0xffff0ff0
>>> dr5		 0x400
>>> dr6	    0xffff0ff0
>>> dr7		 0x400
>>> 0xc0659b37 = kdb_enter+0x2b:	nop
>>> db> show witness
>>> No such command
>>> db> show pcpu
>>> cpuid	     = 1
>>> curthread    = 0xc3c34c00: pid 82885 \"sendmail\"
>>> curpcb	     = 0xf1325d90
>>> fpcurthread  = none
>>> idlethread   = 0xc34d8780: pid 11 \"idle: cpu1\"
>>> APIC ID      = 0
>>> currentldt   = 0x50
>>> db> show pcpuu0
>>> cpuid	     = 0
>>> curthread    = 0xc34d8900: pid 12 \"idle: cpu0\"
>>> curpcb	     = 0xe4f9bd90
>>> fpcurthread  = none
>>> idlethread   = 0xc34d8900: pid 12 \"idle: cpu0\"
>>> APIC ID      = 1
>>> currentldt   = 0x50
>>> db> show pcpu 1
>>> cpuid	     = 1
>>> curthread    = 0xc3c34c00: pid 82885 \"sendmail\"
>>> curpcb	     = 0xf1325d90
>>> fpcurthread  = none
>>> idlethread   = 0xc34d8780: pid 11 \"idle: cpu1\"
>>> APIC ID      = 0
>>> currentldt   = 0x50
>>> db> show pcpu 2
>>> CPU 2 not found
>>> db> print sysctllock
>>> c093ace0
>>> db> ps
>>>  pid	proc	 uid  ppid  pgrp  flag	 stat  wmesg	wchan  cmd
>>> 83305 c43b2830	  0   504   504 0000000 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] cron 83304 c3d91a3c 21802   605   605 0000000 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] perl 83303 c3d12c48	  0   795 83303
>>> 0004002 [SLPQ filelist lock 0xc0938bd0][SLP] uptime 83302 c5b13418	  0
>>> 686   686 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83301
>>> c3ba1c48	  0   680   680 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83300 c40c320c	  0   504   504 0000000 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] cron 83299 c43a2830	  0   504   504 0000000 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] cron 83298 c38dec48	  0   504   504
>>> 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] cron 83297 c3f0020c 21802
>>> 605   605 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 83296
>>> c3c36624 21802   605   605 0000000 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> perl 83295 c406620c 21802   605   605 0000000 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] perl 83294 c38de20c 21802   605   605 0000000 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] perl 83293 c41af418	  0   686   686
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83292 c4244a3c	  0
>>>  680   680 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83291
>>> c40c0624	  0   504   504 0000000 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> cron 83290 c43b1a3c	  0   504   504 0000000 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] cron 83289 c5fffc48	  0   664   664 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83288 c3e9d830	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83287 c3cbbc48	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83286
>>> c43a2000	  0   636   636 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail
>>> 83285 c40c4c48	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83284 c413b624	  0   636   636 0000100 [SLPQ
>>> select 0xc0941aa4][SLP] sendmail 83283 c6003418	  0   664   664 0000100
>>> [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83282 c4140000	  0   650
>>> 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83281 c43ac000
>>>  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail
>>> 83280 c3d91830	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83279 c43afc48	  0   664   664 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83278 c3cbb418	  0   664   664
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83277 c4244c48	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83276
>>> c40c0830	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83275 c4136624	  0   664   664 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83274 c4140a3c	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83273 c6035830	  0   636   636
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83272 c43b120c	  0
>>>  664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83271
>>> c41ada3c	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83270 c3fca624	  0   664   664 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83269 c6035c48	  0   664   664 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83268 c5b13c48	  0   664   664
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83267 c3f82c48	  0
>>>  664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83266
>>> c424320c	  0   664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83265 c43a720c	  0   636   636 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83264 c3e9d418	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83263 c3d18000	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83261 c6036418	  0
>>>  664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83260
>>> c5b0120c	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83259 c4244000	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83258 c4243830	  0   664   664 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83257 c43af830	  0	1 83257
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83255 c5b01830	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83254
>>> c5b01000	  0   650   650 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail
>>> 83248 c3b99c48	  0   680   680 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83247 c42d5c48	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83246 c3f00a3c	  0   664   664
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83245 c6036000	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83244
>>> c40c3c48	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83243 c5fff830	  0   636   636 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83242 c5b01418	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83241 c42a2624	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83240 c41af000	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83239
>>> c41ad830	  0   636   636 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83238 c3d1820c	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83237 c3f00624	  0   664   664 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83236 c5b13a3c	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83235 c42a7a3c	  0
>>>  664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83234
>>> c3f82830	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83233 c43a7624	  0   664   664 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83231 c4376418	  0   683   683 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83230 c406e830	  0   664   664
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83229 c42d520c	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83228
>>> c43af418	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83227 c413ba3c	  0   664   664 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83226 c3e9d20c	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83225 c43a220c	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83224 c3b99418	  0
>>>  636   636 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83223
>>> c3c33c48	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83222 c4375000	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83221 c3e9d624	  0   664   664 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83220 c38de830	  0   636   636
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83219 c3f8020c	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83218
>>> c3f02a3c	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83213 c4375a3c	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83212 c5b09418	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83211 c43af624	  0   664   664
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83208 c4135830	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83207
>>> c41afa3c	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83206 c406ea3c	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83205 c413b418	  0   664   664 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83195 c3f80624	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83193 c5b0920c	  0
>>>  664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83192
>>> c3c33624	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83191 c3c36a3c	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83190 c4140418	  0   636   636 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83189 c4135418	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83188 c437620c	  0
>>>  636   636 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83187
>>> c406e624	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83186 c43a7418	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83185 c3c36c48	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83184 c3cbb830	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83183 c406e000	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83182
>>> c43a2c48	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83181 c40c3000	  0   664   664 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83180 c4140c48	  0   664   664 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83179 c42a2418	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83178 c437e000	  0
>>>  664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83175
>>> c5fff20c	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83174 c3f82624	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83168 c3ba1830	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83167 c4243a3c	  0   664   664
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83164 c6003000	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83162
>>> c4066a3c	  0   664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83157 c40c4000	  0   664   664 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83154 c600320c	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83153 c40c0c48	  0   664   664
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83152 c3d18418	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83150
>>> c41adc48	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83149 c3e9dc48	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83148 c3cba624	  0   664   664 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83146 c3c36000	  0   650   650
>>> 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail 83145 c3f01830	  0   650
>>> 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83142 c41af20c
>>>  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail
>>> 83141 c3e9da3c	  0   664   664 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83140 c5fff624	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83134 c41af624	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83132 c4136830	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83131
>>> c603620c	  0   664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83130 c3f00418	  0   650   650 0000100 [SLPQ select
>>> 0xc0941aa4][SLP] sendmail 83129 c3fcac48	  0   664   664 0000100 [SLPQ
>>> select 0xc0941aa4][SLP] sendmail 83128 c40c4418	  0   650   650 0000100
>>> [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83125 c42d4c48	  0   664
>>> 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83124 c3fca20c
>>>  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail
>>> 83123 c437e418	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83121 c4135c48	  0   664   664 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83119 c41af830	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83118 c38dd000	  0
>>>  650   650 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail 83117 c42d4a3c
>>>  0   664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail
>>> 83114 c42a720c	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83113 c41aac48	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83112 c42d420c	  0   664   664
>>> 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail 83109 c40c4a3c	  0   650
>>> 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83108 c3f8220c
>>>  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail
>>> 83107 c3d10830	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83105 c41aa000	  0   650   650 0000100 [SLPQ
>>> select 0xc0941aa4][SLP][SWAP] sendmail 83101 c4066624	  0   664   664
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83100 c40c3624	  0
>>>  650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83097
>>> c42a2000	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83094 c41aa830	  0   650   650 0000100 [SLPQ select
>>> 0xc0941aa4][SLP] sendmail 83093 c43a7a3c	  0   664   664 0000100 [SLPQ
>>> select 0xc0941aa4][SLP] sendmail 83092 c43b1000	  0   650   650 0000100
>>> [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83091 c437ec48	  0   650
>>> 650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83087 c5fff000
>>>  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail
>>> 83070 c42a2830	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83069 c6036c48	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83056 c6035624	  0   664   664
>>> 0000100 [SLPQ select 0xc0941aa4][SLP][SWAP] sendmail 83051 c41ad20c	  0
>>> 650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 83050
>>> c4244830	  0   664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 83041 c40c0a3c	  0   664   664 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 83038 c424420c	  0   664   664 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 83037 c43a2624	  0   664   664
>>> 0000100 [SLPQ select 0xc0941aa4][SLP][SWAP] sendmail 83034 c413b000	  0
>>> 664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82994
>>> c43b2000 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 82993 c38d8c48 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 82992 c5b01a3c 21802   551   551 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 82991 c5b01c48 21802   550   550 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 82990 c43b2624 21802   551   551 0000000
>>> [SLPQ select 0xc0941aa4][SLP] perl 82989 c5b09000 21802   550   550
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82987 c6036830 21802   551
>>> 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82986 c42d5624 21802   550
>>>   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82984 c5b09624 21802
>>> 551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82983 c5b09830 21802
>>>   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82982 c4135a3c
>>> 21802   551   551 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 82981
>>> c3f82000 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 82976 c42d5830 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 82975 c5b09c48 21802   550   550 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 82974 c43acc48 21802   551   551 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 82973 c3f02418 21802   550   550 0000000
>>> [SLPQ select 0xc0941aa4][SLP] perl 82972 c5b13000 21802   551   551
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82971 c43ac20c 21802   550
>>> 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82970 c3cbb20c 21802   551
>>>   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82969 c43a2418 21802
>>> 550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82967 c43a7000 21802
>>>   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82966 c40c0000
>>> 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82965
>>> c4376c48 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 82964 c43b1c48 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 82962 c41ad624 21802   551   551 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 82961 c38dea3c 21802   550   550 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 82960 c38a820c 21802   551   551 0000000
>>> [SLPQ select 0xc0941aa4][SLP] perl 82959 c43b0c48 21802   550   550
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82952 c42a2a3c 21802   551
>>> 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82951 c6035000 21802   550
>>>   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82948 c43b220c 21802
>>> 551   551 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 82947
>>> c6035a3c 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 82945 c43b2418 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 82944 c41aaa3c 21802   550   550 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 82938 c40c420c 21802   551   551 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 82937 c6036a3c 21802   550   550 0000000
>>> [SLPQ select 0xc0941aa4][SLP] perl 82932 c3f01a3c 21802   551   551
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82931 c42d4624 21802   550
>>> 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82930 c41aa418 21802   551
>>>   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82929 c3d94c48 21802
>>> 550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82928 c3f02830 21802
>>>   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82927 c3d94a3c
>>> 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82926
>>> c413620c 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 82925 c3fcbc48 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 82924 c42d5a3c 21802   551   551 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 82923 c4140830 21802   550   550 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 82922 c43ac830 21802   551   551 0000000
>>> [SLPQ select 0xc0941aa4][SLP] perl 82921 c3f0220c 21802   550   550
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82919 c43b0418	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82918 c3cbba3c
>>> 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82917
>>> c4066830 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 82916 c3ba1624	  0   636   636 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 82915 c4376000 21802   551   551 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 82914 c3f01c48 21802   550   550 0000000
>>> [SLPQ select 0xc0941aa4][SLP] perl 82913 c43b1418	  0   636   636 0000100
>>> [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82909 c4243c48 21802   551
>>>  551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82908 c3c33000 21802
>>> 550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82907 c4243000	  0
>>> 650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82906
>>> c437ea3c 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 82905 c3ba1000 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 82904 c43b1624	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 82903 c40c3a3c 21802   551   551 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 82902 c3f82a3c 21802   550   550 0000000
>>> [SLPQ select 0xc0941aa4][SLP] perl 82896 c43b0830 21802   551   551
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82895 c4136000 21802   550
>>> 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82894 c42a7000 21802   551
>>>   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82893 c3e9d000 21802
>>> 550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82892 c3c3320c 21802
>>>   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82891 c42a220c
>>> 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82890
>>> c4375c48	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 82889 c3ba1a3c	  0   664   664 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 82888 c43af000	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 82887 c42a2c48 21802   551   551
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82886 c3c33418 21802   550
>>> 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82885 c3c36830	  0   636
>>> 636 0000100 [CPU 1] sendmail
>>> 82877 c42a7418 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 82876 c3cbaa3c 21802   550   550 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 82874 c413bc48	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 82870 c43a7c48	  0   650   650
>>> 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82865 c437520c
>>> 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82864
>>> c4376830 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 82851 c3f80418	  0   650   650 0000100 [SLPQ select
>>> 0xc0941aa4][SLP][SWAP] sendmail 82837 c3f02c48	  0   650   650 0000100
>>> [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82782 c42d5000 21802   551
>>>  551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82781 c3d9120c 21802
>>> 550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82769 c43aca3c	  0
>>> 650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82766
>>> c40c020c 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 82765 c41aa20c 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 82763 c3d1220c	  0   664   664 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 82762 c3d9420c 21802   551   551 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 82761 c42a7624 21802   550   550 0000000
>>> [SLPQ select 0xc0941aa4][SLP] perl 82740 c6036624	  0   664   664 0000100
>>> [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82738 c4136a3c 21802   551
>>>  551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82737 c4243418 21802
>>> 550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 82729 c3f82418	  0
>>> 664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 82676
>>> c4376624	  0   664   664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 82621 c3c33a3c	  0   650   650 0000100 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] sendmail 82607 c4376a3c	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 81310 c6035418	  0   664   664
>>> 0000100 [SLPQ select 0xc0941aa4][SLP][SWAP] sendmail 80076 c42d4000	  0
>>> 650   650 0000100 [SLPQ select 0xc0941aa4][SLP][SWAP] sendmail 79627
>>> c4135624	  0   650   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 79198 c3cba830 21802   551   551 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 79197 c414020c 21802   550   550 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 79193 c4066418 21802   551   551 0000000
>>> [SLPQ filelist lock 0xc0938bd0][SLP] perl 79192 c40c3830 21802   550
>>> 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79189 c3f00000 21802   551
>>>   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79188 c437e830 21802
>>> 550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79186 c3d18830 21802
>>>   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79185 c3fca418
>>> 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79184
>>> c4136418 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 79183 c3fcaa3c 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 79179 c3f80000 21802   551   551 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 79178 c3fcb20c 21802   550   550 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 79168 c603520c 21802   551   551 0000000
>>> [SLPQ filelist lock 0xc0938bd0][SLP] perl 79167 c4243624 21802   550
>>> 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 79083 c4375830	  0   664
>>> 664 0000100 [SLPQ sbwait 0xc4401900][SLP][SWAP] sendmail 78838 c3f01000
>>> 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 78837
>>> c3f80830 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 78491 c40c4624 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 78490 c3d94830 21802   550   550 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 78484 c3f02000 21802   551   551 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 78483 c406ec48 21802   550   550 0000000
>>> [SLPQ select 0xc0941aa4][SLP] perl 73847 c43ac418 21802   551   551
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 73846 c42d4418 21802   550
>>> 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 68027 c43b2a3c 21802   550
>>>   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 67937 c3fcb000 21802
>>> 551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 67936 c3fcb624 21802
>>>   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 67933 c42d4830
>>> 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 67932
>>> c38de624 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 67904 c413b830 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 67903 c3cbb624 21802   550   550 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 66287 c43afa3c	  0   650   650 0000100 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] sendmail 60279 c3f00830 21802   551   551
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 60278 c4244624 21802   551
>>> 551 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 60277 c3fcba3c
>>> 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 60276
>>> c3fca000 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 60249 c406e418 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 60248 c3f00c48 21802   550   550 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 51444 c3c3620c 21802   551   551 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 51443 c4066000 21802   550   550 0000000
>>> [SLPQ select 0xc0941aa4][SLP] perl 51430 c3b99624 21802   551   551
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51429 c41aa624 21802   550
>>> 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51427 c40c4830 21802   551
>>>   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51426 c3f0120c 21802
>>> 550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51231 c4375624 21802
>>>   551   551 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 51230
>>> c3fcb830 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 51228 c3d10a3c 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 51227 c4140624 21802   550   550 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 51222 c413b20c 21802   551   551 0000000 [SLPQ
>>> filelist lock 0xc0938bd0][SLP] perl 51221 c38de418 21802   550   550
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51218 c43b020c 21802   551
>>> 551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51217 c3cbac48 21802   550
>>>   550 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 51215 c43a2a3c
>>> 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51214
>>> c437e20c 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 51213 c4244418 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP]
>>> perl 51212 c3f02624 21802   550   550 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] perl 51211 c3f80a3c 21802   551   551 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] perl 51206 c43b0624 21802   550   550 0000000
>>> [SLPQ select 0xc0941aa4][SLP] perl 51161 c413520c 21802   551   551
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51160 c42d5418 21802   550
>>> 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 51153 c40c0418 21802   551
>>>   551 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 50987 c3c36418
>>> 0	1 50987 0000100 [SLPQ sbwait 0xc3da3c40][SLP] sendmail 36954 c4135000
>>> 21802   550   550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 36941
>>> c38dd418 21802   551   551 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 32803 c43af20c	  0 32793 32803 0004003 [SLPQ ttyin 0xc3871810][SLP][SWAP]
>>> zsh 32793 c43b2c48	127 32792 32793 0004002 [SLPQ pause
>>> 0xc43b2c7c][SLP][SWAP] zsh 32792 c42a7c48	127 32781 32781 0000100 [SLPQ
>>> select 0xc0941aa4][SLP][SWAP] sshd 32781 c3fcb418	  0   491 32781 0004100
>>> [SLPQ sbwait 0xc4547e90][SLP][SWAP] sshd 31218 c3cbb000 21802   551   551
>>> 0000000 [SLPQ select 0xc0941aa4][SLP] perl 31217 c3f01418 21802   550
>>> 550 0000000 [SLPQ select 0xc0941aa4][SLP] perl 5491 c43b0a3c	  0	1  5491
>>> 0000100 [SLPQ sbwait 0xc45473e8][SLP] sendmail 795 c3d91c48	  0   778
>>> 795 0004002 [SLPQ wait 0xc3d91c48][SLP] sh 778 c3d94418	  0   760   778
>>> 0004002 [SLPQ pause 0xc3d9444c][SLP][SWAP] zsh 760 c3ea1000	  0	1   760
>>> 0004102 [SLPQ wait 0xc3ea1000][SLP][SWAP] login 759 c3ea120c	  0	1   759
>>> 0004002 [SLPQ ttyin 0xc3737410][SLP][SWAP] getty 758 c3ea1418	  0	1   758
>>> 0004002 [SLPQ ttyin 0xc3734c10][SLP][SWAP] getty 757 c3ea1624	  0	1   757
>>> 0004002 [SLPQ ttyin 0xc3734410][SLP][SWAP] getty 756 c3ea1830	  0	1   756
>>> 0004002 [SLPQ ttyin 0xc372f410][SLP][SWAP] getty 755 c3ea1a3c	  0	1   755
>>> 0004002 [SLPQ ttyin 0xc3736410][SLP][SWAP] getty 754 c3d18c48	  0	1   754
>>> 0004002 [SLPQ ttyin 0xc3730c10][SLP][SWAP] getty 753 c3d18624	  0	1   753
>>> 0004002 [SLPQ ttyin 0xc3735810][SLP][SWAP] getty 752 c38de000	  0	1   752
>>> 0004002 [SLPQ ttyin 0xc3734810][SLP][SWAP] getty 719 c3d91418	  0	1   717
>>> 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] snmpd 709 c3d18a3c	  0	1
>>> 709 0000100 [SLPQ pause 0xc3d18a70][SLP][SWAP] sendmail 706 c3d12418
>>> 0	1   706 0000100 [SLPQ pause 0xc3d1244c][SLP][SWAP] sendmail 703
>>> c3d91000	  0	1   703 0000100 [SLPQ pause 0xc3d91034][SLP][SWAP] sendmail
>>> 700 c3d91624	  0	1   700 0000100 [SLPQ pause 0xc3d91658][SLP][SWAP]
>>> sendmail 696 c3d12a3c	  0	1   696 0000100 [SLPQ pause
>>> 0xc3d12a70][SLP][SWAP] sendmail 692 c3d94000	  0	1   692 0000100 [SLPQ
>>> pause 0xc3d94034][SLP][SWAP] sendmail 689 c3d94624	  0	1   689 0000100
>>> [SLPQ pause 0xc3d94658][SLP][SWAP] sendmail 686 c3d1020c	  0	1   686
>>> 0000100 [SLPQ pause 0xc3d10240][SLP] sendmail 683 c3d10624	  0	1   683
>>> 0000100 [SLPQ pause 0xc3d10658][SLP][SWAP] sendmail 680 c3d12000	  0	1
>>> 680 0000100 [SLPQ pause 0xc3d12034][SLP] sendmail 678 c3d12624	  0	1
>>> 678 0000100 [SLPQ select 0xc0941aa4][SLP] sendmail 664 c3d10000	  0	1
>>> 664 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 650 c3cba000
>>> 0	1   650 0000100 [SLPQ filelist lock 0xc0938bd0][SLP] sendmail 636
>>> c3d10418	  0	1   636 0000100 [SLPQ filelist lock 0xc0938bd0][SLP]
>>> sendmail 620 c38d8418	  0   616   616 0000001 [SLPQ lockf
>>> 0xc41d3bc0][SLP][SWAP] saslauthd 619 c3b99830	  0   616   616 0000001
>>> [SLPQ lockf 0xc878a040][SLP][SWAP] saslauthd 618 c3cba418	  0   616   616
>>> 0000001 [SLPQ lockf 0xc44dc900][SLP][SWAP] saslauthd 617 c38d8624	  0
>>> 616   616 0000001 [SLPQ accept 0xc3cf0466][SLP][SWAP] saslauthd 616
>>> c3d10c48	  0	1   616 0000001 [SLPQ lockf 0xc44dc380][SLP][SWAP] saslauthd
>>> 605 c3cba20c 21802	 1   605 0000000 [SLPQ select 0xc0941aa4][SLP] perl
>>> 604 c3d12830	  0   577   604 0004003 [SLPQ ttyin 0xc3725810][SLP][SWAP]
>>> zsh 577 c3c33830	127   576   577 0004002 [SLPQ pause
>>> 0xc3c33864][SLP][SWAP] zsh 576 c3b9920c	127   574   574 0000100 [SLPQ
>>> select 0xc0941aa4][SLP][SWAP] sshd 574 c38dd624	  0   491   574 0004100
>>> [SLPQ sbwait 0xc3c30638][SLP][SWAP] sshd 551 c38a5a3c 21802	 1   551
>>> 0000001 [SLPQ select 0xc0941aa4][SLP] perl 550 c38ddc48 21802	 1   550
>>> 0000000 [SLPQ filelist lock 0xc0938bd0][SLP] perl 546 c3b99a3c 21802	 1
>>> 546 0000001 [SLPQ select 0xc0941aa4][SLP] perl 539 c38a5830 21802   534
>>> 534 0000000 [SLPQ select 0xc0941aa4][SLP][SWAP] httpd 538 c38a8000 21802
>>>  534   534 0000000 [SLPQ select 0xc0941aa4][SLP][SWAP] httpd 537 c38d820c
>>> 21802   534   534 0000000 [SLPQ select 0xc0941aa4][SLP][SWAP] httpd 536
>>> c3ba120c 21802   534   534 0000000 [SLPQ select 0xc0941aa4][SLP][SWAP]
>>> httpd 535 c3ba1418 21802   534   534 0000000 [SLPQ select
>>> 0xc0941aa4][SLP][SWAP] httpd 534 c38d8000 21802	 1   534 0000000 [SLPQ
>>> select 0xc0941aa4][SLP] httpd 527 c38a5624	  0	1   527 0008180 (threaded)
>>>  perl
>>>   thread 0xc41c1a80 ksegrp 0xc34db660 [SLPQ kserel 0xc34db694][SLP]
>>>   thread 0xc879b300 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc43a6a80 ksegrp 0xc34db660 [SLPQ kserel
>>> 0xc34db694][SLP] thread 0xc42eec00 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a8480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59b5600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e8480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4ade000 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42ee480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e5180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a9300 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a2480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc423a600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc44f2600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5a0b780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5a36600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e9780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42ee780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e9300 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e7a80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5a20000 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc87e9c00 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc42efa80 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc41c0780 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc4099900 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc43c9780 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc43ca180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41c4180 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc41c1600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59e7c00 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a2a80 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc5a20d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422d480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5a20600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3ea2900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41a1180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a3780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4096480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59e7480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a6900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc442dc00 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3f03000 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5a20a80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4adf180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc599d480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42ef600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc442e600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4ade480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e7180 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc879ba80 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc59b5180 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc4120d80 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc5a0b900 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc87e7600 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc599d900 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc423a900 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc3ea2600 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc87e8780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422d780 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc5a20900 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc5a36480 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc43ca300 ksegrp 0xc34db660 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc41a1600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3c35780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3d96900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87ead80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4067000 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3f28180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4ade600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc8a96000 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e5900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422d180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e9900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e7480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc44ef180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59b5900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e5d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc599d180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59b5000 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4ade780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a4480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4121300 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3f92900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422d600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4adf000 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3f8fa80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a8780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4adea80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3ea2480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc423aa80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4adf780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e5300 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3f03180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4099300 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc442d480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4068780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4121d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a4780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4095d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a2000 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a4900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43c9d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e6d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc8a96600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43ca480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59b5d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59e7300 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3d17600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e8c00 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42f1600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a6780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5987c00 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422c600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc442d300 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc8a97000 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3ea3c00 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a8a80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc44efd80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87ea780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5987000 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4ade900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41c4300 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5a20c00 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e6900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3d17480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4ae1c00 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43cad80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87ea900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422dd80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e7d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e9180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4adf900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4aeb780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5a20780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a2180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59e7d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43c9180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42f1480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5987600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41c1780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a4d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41c1180 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc44ef480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4ae1a80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5987480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a3900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e9480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc44ef900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3d95d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc599d600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5987900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422cc00 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc599d000 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3ea2300 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e6480 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42f1d80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e8600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4120780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc599d780 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41bfc00 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4aeb900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42ee900 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4067a80 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a8600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4121600 ksegrp 0xc34db660 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc38df900 ksegrp 0xc354d8a0 [SLPQ ksesigwait
>>> 0xc38a575c][SLP] 520 c38dd830 21803	 1    69 000c083 (threaded)  perl
>>>   thread 0xc43a9600 ksegrp 0xc354dcc0 [SLPQ kserel 0xc354dcf4][SLP]
>>>   thread 0xc8a97a80 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP]
>>>   thread 0xc41a0c00 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP]
>>>   thread 0xc5a0bc00 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP]
>>>   thread 0xc87e9d80 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP]
>>>   thread 0xc43a8900 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP]
>>>   thread 0xc59e7000 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP]
>>>   thread 0xc3f8f180 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP]
>>>   thread 0xc8a97d80 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP]
>>>   thread 0xc599dc00 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP]
>>>   thread 0xc4ae1480 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP]
>>>   thread 0xc42ee300 ksegrp 0xc354dcc0 [SLPQ kserel 0xc354dcf4][SLP]
>>>   thread 0xc87e8a80 ksegrp 0xc354dcc0 [SLPQ select 0xc0941aa4][SLP]
>>>   thread 0xc87e5600 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc87e7300 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc5a0bd80 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc442da80 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc879b480 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc879b780 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc44f2180 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc3f90780 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc5987d80 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc879bc00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e7c00 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc42ee180 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc87a2300 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc59b5480 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc442e900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5987180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5987780 ksegrp 0xc354dcc0 [SLPQ filelist lock
>>> 0xc0938bd0][SLP] thread 0xc5a0b480 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc599dd80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87ea180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3c34780 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59b5c00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41c0900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a9900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3f92c00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a3480 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e9000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3f81d80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41a0d80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5a36000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a9480 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc8a96a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42f1900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a2d80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41c0600 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4120000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e8900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e7780 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a3300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e9a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc423a480 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4099d80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5a20480 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc442d180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59e7180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4096d80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43ca600 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5a36180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc423ad80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc442ec00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41c4c00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e5c00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4096a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3ea3600 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41c4000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a4c00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43c9300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42eed80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a9a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4adfd80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4ade180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41a1480 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a4000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5987a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e6a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a9780 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc8a96180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4aeb000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42ef480 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4120a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4121180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc44ef300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a3a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc8a96780 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59b5780 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a9180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e8180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4adf600 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc423a300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41bfd80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3ea3480 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422c780 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422c300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422d900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc44f2000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc8a97600 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59e7900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3f28780 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4aeb180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e8d80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a8300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3f28c00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43c9600 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43c9900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41c0a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41bf600 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422c900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43c9a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42f1a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3ea3900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43caa80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43ca900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43c9c00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc442d600 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc8a96300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87ea480 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc43a8180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41c1c00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3b9a000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41bf180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4121c00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4aebc00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422da80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e5000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc422c180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87eaa80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42ef300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e6000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42f1c00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59e7a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc41c0d80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4adfc00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc442e780 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4099000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87a4300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4121000 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc8a97180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4121480 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc59b5300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc879b600 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc87e9600 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3f90c00 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc8a96d80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc8a97300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4122a80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc42eea80 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc5987300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc423a780 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4068300 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4ae1180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc3f8f480 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc442e180 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc4ae1900 ksegrp 0xc354dcc0 [SLPQ select
>>> 0xc0941aa4][SLP] thread 0xc38e0180 ksegrp 0xc354d900 [SLPQ ksesigwait
>>> 0xc38dd968][SLP] 519 c38a5c48	  0	1    69 0004002 [SLPQ sbwait
>>> 0xc39cabc8][SLP][SWAP] perl 504 c38a5418	  0	1   504 0000000 [SLPQ nanslp
>>> 0xc093af0c][SLP] cron 491 c38a5000	  0	1   491 0000100 [SLPQ filelist
>>> lock 0xc0938bd0][SLP] sshd 474 c38dd20c	  0	1   474 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] ntpd 381 c38a520c	  0	1   381 0000000 [SLPQ select
>>> 0xc0941aa4][SLP] syslogd 352 c38d8830	  0	1   352 0000000 [SLPQ select
>>> 0xc0941aa4][SLP][SWAP] devd 268 c38dda3c	 64   262   262 0000100 [SLPQ
>>> bpf 0xc38dce00][SLP] pflogd 262 c38d8a3c	  0	1   262 0000000 [SLPQ sbwait
>>> 0xc39ca370][SLP][SWAP] pflogd 68 c38a8418	  0	0     0 0000204 [SLPQ -
>>> 0xef03ed08][SLP] schedcpu 67 c38a8624	  0	0     0 0000204 [SLPQ -
>>> 0xc094a02c][SLP] nfsiod 3 66 c38a8830	  0	0     0 0000204 [SLPQ -
>>> 0xc094a028][SLP] nfsiod 2 65 c38a8a3c	  0	0     0 0000204 [SLPQ -
>>> 0xc094a024][SLP] nfsiod 1 64 c38a8c48	  0	0     0 0000204 [SLPQ -
>>> 0xc094a020][SLP] nfsiod 0 63 c355a20c	  0	0     0 0000204 [SLPQ vlruwt
>>> 0xc355a20c][SLP] vnlru 62 c355a418	  0	0     0 0000204 [SLPQ syncer
>>> 0xc093ac7c][SLP] syncer 61 c355a624	  0	0     0 0000204 [SLPQ psleep
>>> 0xc0941fec][SLP] bufdaemon 60 c355a830	  0	0     0 000020c [SLPQ pgzero
>>> 0xc09505e4][SLP] pagezero 59 c355aa3c	  0	0     0 0000204 [SLPQ psleep
>>> 0xc0950134][SLP] vmdaemon 58 c355ac48	  0	0     0 0000204 [SLPQ psleep
>>> 0xc09500f0][SLP] pagedaemon 57 c3603000	  0	0     0 0000204 [IWAIT] swi0:
>>> sio
>>>   56 c360320c	  0	0     0 0000204 [SLPQ - 0xc372903c][SLP] fdc0
>>>   55 c3603418	  0	0     0 0000204 [SLPQ aifthd 0xc3603418][SLP] aac0aif
>>>   54 c3603624	  0	0     0 0000204 [SLPQ idle 0xc3528c00][SLP]
>>> aic_recovery0 53 c3603830	  0	0     0 0000204 [SLPQ idle 0xc3528c00][SLP]
>>> aic_recovery0 52 c3603a3c	  0	0     0 0000204 [IWAIT] swi6:+
>>>   51 c3603c48	  0	0     0 0000204 [IWAIT] swi6: task queue
>>>    9 c354b624	  0	0     0 0000204 [SLPQ - 0xc3563600][SLP] acpi_task2
>>>    8 c354b830	  0	0     0 0000204 [SLPQ - 0xc3563600][SLP] acpi_task1
>>>    7 c354ba3c	  0	0     0 0000204 [SLPQ - 0xc3563600][SLP] acpi_task0
>>>   50 c354bc48	  0	0     0 0000204 [IWAIT] swi2: cambio
>>>    6 c3559000	  0	0     0 0000204 [SLPQ - 0xc3563780][SLP] kqueue taskq
>>>   49 c355920c	  0	0     0 0000204 [IWAIT] swi5:+
>>>    5 c3559418	  0	0     0 0000204 [SLPQ - 0xc3563980][SLP] thread taskq
>>>   48 c3559624	  0	0     0 0000204 [SLPQ - 0xc0935c60][SLP] yarrow
>>>    4 c3559830	  0	0     0 0000204 [SLPQ - 0xc0938428][SLP] g_down
>>>    3 c3559a3c	  0	0     0 0000204 [SLPQ - 0xc0938424][SLP] g_up
>>>    2 c3559c48	  0	0     0 0000204 [SLPQ - 0xc093841c][SLP] g_event
>>>   47 c355a000	  0	0     0 0000204 [IWAIT] swi1: net
>>>   46 c353bc48	  0	0     0 0000204 [IWAIT] swi3: vm
>>>   45 c354a000	  0	0     0 000020c [IWAIT] swi4: clock sio
>>>   44 c354a20c	  0	0     0 0000204 [IWAIT] irq0:
>>>   43 c354a418	  0	0     0 0000204 [IWAIT] irq31: aac0
>>>   42 c354a624	  0	0     0 0000204 [IWAIT] irq30: ahc0
>>>   41 c354a830	  0	0     0 0000204 [IWAIT] irq29:
>>>   40 c354aa3c	  0	0     0 0000204 [IWAIT] irq28:
>>>   39 c354ac48	  0	0     0 0000204 [IWAIT] irq27:
>>>   38 c354b000	  0	0     0 0000204 [IWAIT] irq26:
>>>   37 c354b20c	  0	0     0 0000204 [IWAIT] irq25:
>>>   36 c354b418	  0	0     0 0000204 [IWAIT] irq24:
>>>   35 c3529624	  0	0     0 0000204 [IWAIT] irq23:
>>>   34 c3529830	  0	0     0 0000204 [IWAIT] irq22: fxp0
>>>   33 c3529a3c	  0	0     0 0000204 [IWAIT] irq21:
>>>   32 c3529c48	  0	0     0 0000204 [IWAIT] irq20:
>>>   31 c353b000	  0	0     0 0000204 [IWAIT] irq19:
>>>   30 c353b20c	  0	0     0 0000204 [IWAIT] irq18: fxp1
>>>   29 c353b418	  0	0     0 0000204 [IWAIT] irq17: bge0
>>>   28 c353b624	  0	0     0 0000204 [IWAIT] irq16: fxp2
>>>   27 c353b830	  0	0     0 0000204 [IWAIT] irq15: ata1
>>>   26 c353ba3c	  0	0     0 0000204 [IWAIT] irq14: ata0
>>>   25 c34dc20c	  0	0     0 0000204 [IWAIT] irq13:
>>>   24 c34dc418	  0	0     0 0000204 [IWAIT] irq12:
>>>   23 c34dc624	  0	0     0 0000204 [IWAIT] irq11:
>>>   22 c34dc830	  0	0     0 0000204 [IWAIT] irq10:
>>>   21 c34dca3c	  0	0     0 0000204 [IWAIT] irq9: acpi0
>>>   20 c34dcc48	  0	0     0 0000204 [IWAIT] irq8:
>>>   19 c3529000	  0	0     0 0000204 [IWAIT] irq7: ppc0
>>>   18 c352920c	  0	0     0 0000204 [IWAIT] irq6: fdc0
>>>   17 c3529418	  0	0     0 0000204 [IWAIT] irq5:
>>>   16 c34d7000	  0	0     0 0000204 [IWAIT] irq4: sio0
>>>   15 c34d720c	  0	0     0 0000204 [IWAIT] irq3: sio1
>>>   14 c34d7418	  0	0     0 0000204 [IWAIT] irq2:
>>>   13 c34d7624	  0	0     0 0000204 [IWAIT] irq1: atkbd0
>>>   12 c34d7830	  0	0     0 000020c [CPU 0] idle: cpu0
>>>   11 c34d7a3c	  0	0     0 000020c [Can run] idle: cpu1
>>>    1 c34d7c48	  0	0     1 0004200 [SLPQ wait 0xc34d7c48][SLP] init
>>>   10 c34dc000	  0	0     0 0000204 [SLPQ ktrace 0xc0938e78][SLP] ktrace
>>>    0 c0938520	  0	0     0 0000200 [IWAIT] swapper
>>> db> panic
>>> panic: from debugger
>>> cpuid = 1
>>> KDB: stack backtrace:
>>> kdb_backtrace(100,c3c34c00,0,c089e89c,c089e8b8) at 0xc0659ab5 =
>>> kdb_backtrace+0x29 panic(c084a8b4,f1325a04,c0468c70,c093ace0,0) at
>>> 0xc06408b8 = panic+0x114 db_panic(c093ace0,0,ffffffff,f1325978,0) at
>>> 0xc0468cd9 = db_panic+0xd
>>> db_command(c09217e4,c08a7820,c089e89c,c089e8b8,c084a8c2) at 0xc0468c70 =
>>> db_command+0x264 db_command_loop(0,0,f1325a30,f1325a1c,f1325a64) at
>>> 0xc0468d38 = db_command_loop+0x5c db_trap(3,0,c3c34c00,3,c3c34c00) at
>>> 0xc046a949 = db_trap+0xdd
>>> kdb_trap(3,0,f1325abc) at 0xc0659dd0 = kdb_trap+0x88
>>> trap(f1320008,c0650028,c0870028,f9,c372ec00) at 0xc0817c88 = trap+0x498
>>> calltrap() at 0xc0804cba = calltrap+0x5
>>> --- trap 0x3, eip = 0xc0659b37, esp = 0xf1325afc, ebp = 0xf1325afc ---
>>> kdb_enter(c0895a5e) at 0xc0659b37 = kdb_enter+0x2b
>>> siointr1(c372ec00) at 0xc07f09b2 = siointr1+0xce
>>> siointr(c372ec00) at 0xc07f0792 = siointr+0x5e
>>> intr_execute_handlers(c3528090,f1325b68,4,f1325bcc,c0805073) at
>>> 0xc0808985 = intr_execute_handlers+0x85 lapic_handle_intr(34) at
>>> 0xc080b042 = lapic_handle_intr+0x2e
>>> Xapic_isr1() at 0xc0805073 = Xapic_isr1+0x33
>>> --- interrupt, eip = 0xc0685d59, esp = 0xf1325bac, ebp = 0xf1325bcc ---
>>> unp_gc(c43be2c8) at 0xc0685d59 = unp_gc+0xf1
>>> unp_detach(c43b7000,f1325c08,c067a188,c43be2c8,0) at 0xc06840fa =
>>> unp_detach+0xae uipc_detach(c43be2c8) at 0xc0682f8e = uipc_detach+0x82
>>> soclose(c43be2c8,c40fb6c0,0,f1325c34,c0620c10) at 0xc067a188 =
>>> soclose+0x278 soo_close(c40fb6c0,c3c34c00) at 0xc0669fb3 = soo_close+0x63
>>> fdrop_locked(c40fb6c0,c3c34c00,c3a3d800,f1325cb4,c061f0fb) at 0xc0620c10
>>> = fdrop_locked+0xac fdrop(c40fb6c0,c3c34c00,c066371f,c40fb6c0,c878ac00)
>>> at 0xc0620b5d = fdrop+0x41 closef(c40fb6c0,c3c34c00,0,c3c34c00,6) at
>>> 0xc061f0fb = closef+0x427 close(c3c34c00,f1325d04,1,0,292) at 0xc061c4bd
>>> = close+0x211
>>> syscall(3b,1003b,bfbf003b,80d58e7,80e8780) at 0xc0818557 = syscall+0x2bf
>>> Xint0x80_syscall() at 0xc0804d0f = Xint0x80_syscall+0x1f
>>> --- syscall (6, FreeBSD ELF32, close), eip = 0x282cb7d3, esp =
>>> 0xbfbfae4c, ebp = 0xbfbfae68 --- KDB: stack backtrace:
>>> kdb_backtrace(28d5a61e,53755e05,ffc03014,c3c34c00,c084a8b4) at 0xc0659ab5
>>> = kdb_backtrace+0x29 mi_switch(1,0,f1325924,c0640190,c3c34c00) at
>>> 0xc0647162 = mi_switch+0xf6 sched_bind(c3c34c00,0) at 0xc065287c =
>>> sched_bind+0x44
>>> boot(104,104,c3c34c00,0,c089e89c) at 0xc0640190 = boot+0x64
>>> panic(c084a8b4,f1325a04,c0468c70,c093ace0,0) at 0xc064095d = panic+0x1b9
>>> db_panic(c093ace0,0,ffffffff,f1325978,0) at 0xc0468cd9 = db_panic+0xd
>>> db_command(c09217e4,c08a7820,c089e89c,c089e8b8,c084a8c2) at 0xc0468c70 =
>>> db_command+0x264 db_command_loop(0,0,f1325a30,f1325a1c,f1325a64) at
>>> 0xc0468d38 = db_command_loop+0x5c db_trap(3,0,c3c34c00,3,c3c34c00) at
>>> 0xc046a949 = db_trap+0xdd
>>> kdb_trap(3,0,f1325abc) at 0xc0659dd0 = kdb_trap+0x88
>>> trap(f1320008,c0650028,c0870028,f9,c372ec00) at 0xc0817c88 = trap+0x498
>>> calltrap() at 0xc0804cba = calltrap+0x5
>>> --- trap 0x3, eip = 0xc0659b37, esp = 0xf1325afc, ebp = 0xf1325afc ---
>>> kdb_enter(c0895a5e) at 0xc0659b37 = kdb_enter+0x2b
>>> siointr1(c372ec00) at 0xc07f09b2 = siointr1+0xce
>>> siointr(c372ec00) at 0xc07f0792 = siointr+0x5e
>>> intr_execute_handlers(c3528090,f1325b68,4,f1325bcc,c0805073) at
>>> 0xc0808985 = intr_execute_handlers+0x85 lapic_handle_intr(34) at
>>> 0xc080b042 = lapic_handle_intr+0x2e
>>> Xapic_isr1() at 0xc0805073 = Xapic_isr1+0x33
>>> --- interrupt, eip = 0xc0685d59, esp = 0xf1325bac, ebp = 0xf1325bcc ---
>>> unp_gc(c43be2c8) at 0xc0685d59 = unp_gc+0xf1
>>> unp_detach(c43b7000,f1325c08,c067a188,c43be2c8,0) at 0xc06840fa =
>>> unp_detach+0xae uipc_detach(c43be2c8) at 0xc0682f8e = uipc_detach+0x82
>>> soclose(c43be2c8,c40fb6c0,0,f1325c34,c0620c10) at 0xc067a188 =
>>> soclose+0x278 soo_close(c40fb6c0,c3c34c00) at 0xc0669fb3 = soo_close+0x63
>>> fdrop_locked(c40fb6c0,c3c34c00,c3a3d800,f1325cb4,c061f0fb) at 0xc0620c10
>>> = fdrop_locked+0xac fdrop(c40fb6c0,c3c34c00,c066371f,c40fb6c0,c878ac00)
>>> at 0xc0620b5d = fdrop+0x41 closef(c40fb6c0,c3c34c00,0,c3c34c00,6) at
>>> 0xc061f0fb = closef+0x427 close(c3c34c00,f1325d04,1,0,292) at 0xc061c4bd
>>> = close+0x211
>>> syscall(3b,1003b,bfbf003b,80d58e7,80e8780) at 0xc0818557 = syscall+0x2bf
>>> Xint0x80_syscall() at 0xc0804d0f = Xint0x80_syscall+0x1f
>>> --- syscall (6, FreeBSD ELF32, close), eip = 0x282cb7d3, esp =
>>> 0xbfbfae4c, ebp = 0xbfbfae68 --- [Previous attempts at using "call
>>> boot(0)"/etc. produced a solid hang that required physical access to the
>>> Reset button / power switch.]
>>> db> call cpu_reset()
>>> cpu_reset: Restarting BSP
>>> cpu_reset_proxy: Stopped CPU 1
>>>
>>>
>>> [Seeing other posts that request kernel config additions to assist with
>>> debugging, I then tried to install a kernel with the added config of:
>>> options WITNESS
>>> and booting failed as such:]
>>>
>>> /boot/kernel/acpi.ko text=0x40c0c data=0x2160+0x1090
>>> syms=[0x4+0x7810+0x4+0xa292] KDB: debugger backends: ddb
>>> KDB: current backend: ddb
>>> Copyright (c) 1992-2005 The FreeBSD Project.
>>> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>>> 	The Regents of the University of California. All rights reserved.
>>> FreeBSD 6.0-RC1 #1: Wed Oct 19 09:45:23 CDT 2005
>>>    root_at_[host-elided]:/usr/obj/usr/src/sys/SMP_DDB
>>> WARNING: WITNESS option enabled, expect reduced performance.
>>> Timecounter \"i8254\" frequency 1193182 Hz quality 0
>>> CPU: Intel Pentium III (997.46-MHz 686-class CPU)
>>>  Origin = \"GenuineIntel\"  Id = 0x68a  Stepping = 10
>>>
>>> Features=0x383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,M
>>> CA,CMOV,PAT,PSE36,MMX,FXSR,SSE> real memory  = 2147418112 (2047 MB)
>>> avail memory = 2096447488 (1999 MB)
>>> ACPI APIC Table: <DELL	 PE2550  >
>>> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
>>> cpu0 (BSP): APIC ID:  1
>>> cpu1 (AP): APIC ID:  0
>>> ioapic0: Changing APIC ID to 2
>>> ioapic1: Changing APIC ID to 3
>>> MADT: Forcing active-low polarity and level trigger for SCI
>>> ioapic0 <Version 1.1> irqs 0-15 on motherboard
>>> ioapic1 <Version 1.1> irqs 16-31 on motherboard
>>> panic: spin lock hptlock not in order list
>>> cpuid = 0
>>> KDB: stack backtrace:
>>> kdb_backtrace(100,c0903380,0,c085e142,c08a28ac) at 0xc0646395 =
>>> kdb_backtrace+0x29 panic(c084621f,c085e142,c090c6c0,0,c0845c3a) at
>>> 0xc062fd4c = panic+0x114 enroll(c085e142,c08a28ac,c096d720,1,c0c20d5c) at
>>> 0xc064f61b = enroll+0xef
>>> witness_init(c096d720,c34bfb74,c08dd510,c28000,c0c20d74) at 0xc064e6dd =
>>> witness_init+0x11d mtx_init(c096d720,c085e142,0,1,c0c20d88) at 0xc0628d63
>>> = mtx_init+0x8f hpt_init(0,c1ec00,c1e000,0,c04457e5) at 0xc07ac352 =
>>> hpt_init+0x16 mi_startup() at 0xc060b1be = mi_startup+0x96
>>> begin() at 0xc04457e5 = begin+0x2c
>>> KDB: enter: panic
>>> [thread pid 0 tid 0 ]
>>> Stopped at	0xc0646417 = kdb_enter+0x2b:	nop
>>> db>
>>> db> where
>>> Tracing pid 0 tid 0 td 0xc0903380
>>> kdb_enter(c0843318) at 0xc0646417 = kdb_enter+0x2b
>>> panic(c084621f,c085e142,c090c6c0,0,c0845c3a) at 0xc062fd5f = panic+0x127
>>> enroll(c085e142,c08a28ac,c096d720,1,c0c20d5c) at 0xc064f61b = enroll+0xef
>>> witness_init(c096d720,c34bfb74,c08dd510,c28000,c0c20d74) at 0xc064e6dd =
>>> witness_init+0x11d mtx_init(c096d720,c085e142,0,1,c0c20d88) at 0xc0628d63
>>> = mtx_init+0x8f hpt_init(0,c1ec00,c1e000,0,c04457e5) at 0xc07ac352 =
>>> hpt_init+0x16 mi_startup() at 0xc060b1be = mi_startup+0x96
>>> begin() at 0xc04457e5 = begin+0x2c
>>> db> show pcpu
>>> cpuid	     = 0
>>> curthread    = 0xc0903380: pid 0 \"swapper\"
>>> curpcb	     = 0xc0c20d90
>>> fpcurthread  = none
>>> idlethread   = 0xc34d8900: pid 12 \"idle: cpu0\"
>>> APIC ID      = 1
>>> currentldt   = 0x50
>>> spin locks held:
>>> db> show pcpu 0
>>> cpuid	     = 0
>>> curthread    = 0xc0903380: pid 0 \"swapper\"
>>> curpcb	     = 0xc0c20d90
>>> fpcurthread  = none
>>> idlethread   = 0xc34d8900: pid 12 \"idle: cpu0\"
>>> APIC ID      = 1
>>> currentldt   = 0x50
>>> spin locks held:
>>> db> show pcpu 1
>>> cpuid	     = 1
>>> curthread    = none
>>> curpcb	     = 0
>>> fpcurthread  = none
>>> idlethread   = 0xc34d8780: pid 11 \"idle: cpu1\"
>>> APIC ID      = 0
>>> currentldt   = 0x50
>>> spin locks held:
>>> db> show sysctllock
>>> No such command
>>> db> print sysctllock
>>> c0905920
>>> db> show reg
>>> cs		  0x20
>>> ds	    0xc0840028
>>> es	    0xc0640028	blist_create+0x4
>>> fs	    0xc0c20008
>>> ss		  0x28
>>> eax		  0x12
>>> ecx	    0xc1033000
>>> edx		     0
>>> ebx	    0xc0c20d0c
>>> esp	    0xc0c20ce0
>>> ebp	    0xc0c20ce0
>>> esi		   0x1
>>> edi	    0xc084621f
>>> eip	    0xc0646417	kdb_enter+0x2b
>>> efl	       0x80292
>>> dr0		     0
>>> dr1		     0
>>> dr2		     0
>>> dr3		     0
>>> dr4	    0xffff0ff0
>>> dr5		 0x400
>>> dr6	    0xffff0ff0
>>> dr7		 0x400
>>> 0xc0646417 = kdb_enter+0x2b:	nop
>>> db> show witness
>>> Sleep locks:
>>> 0 module subsystem sx lock -- last acquired _at_
>>> /usr/src/sys/kern/kern_module.c:115 0 kernel environment -- last acquired
>>> _at_ /usr/src/sys/kern/subr_hints.c:117 0 user map -- last acquired _at_
>>> /usr/src/sys/vm/vm_map.c:2489
>>> 1  system map -- last acquired _at_ /usr/src/sys/vm/vm_kern.c:295
>>> 2   KMAP ENTRY -- last acquired _at_ /usr/src/sys/vm/uma_core.c:2631
>>> 3    UMA zone -- last acquired _at_ /usr/src/sys/vm/uma_core.c:1898
>>> 2   kmem object -- last acquired _at_ /usr/src/sys/vm/vm_kern.c:397
>>> 3    SYSMAPS -- last acquired _at_ /usr/src/sys/i386/i386/pmap.c:2410
>>> 3    vm page queue mutex -- last acquired _at_ /usr/src/sys/vm/vm_kern.c:404
>>> 4     pmap -- last acquired _at_ /usr/src/sys/i386/i386/pmap.c:1878
>>> 4     vnode interlock -- last acquired _at_ order list:0
>>> 5      cdev -- last acquired _at_ /usr/src/sys/kern/kern_conf.c:62
>>> 2   kernel object -- last acquired _at_ /usr/src/sys/vm/vm_object.c:454
>>> 3    SYSMAPS -- (already displayed)
>>> 3    vm page queue mutex -- (already displayed)
>>> 2   UMA lock -- last acquired _at_ /usr/src/sys/vm/uma_core.c:914
>>> 3    UMA zone -- (already displayed)
>>> 0 nfsd_mtx -- last acquired _at_ order list:0
>>> 3  so_snd -- last acquired _at_ order list:0
>>> 4   so_rcv -- last acquired _at_ order list:0
>>> 5    radix node head -- last acquired _at_ order list:0
>>> 6     rtentry -- last acquired _at_ order list:0
>>> 7      ifaddr -- last acquired _at_ order list:0
>>> 5    sellck -- last acquired _at_ order list:0
>>> 0 bpf global lock -- last acquired _at_ order list:0
>>> 1  bpf interface lock -- last acquired _at_ order list:0
>>> 2   bpf cdev lock -- last acquired _at_ order list:0
>>> 0 ddp_list_mtx -- last acquired _at_ order list:0
>>> 1  ddp_mtx -- last acquired _at_ order list:0
>>> 0 slip_mtx -- last acquired _at_ order list:0
>>> 1  slip sc_mtx -- last acquired _at_ order list:0
>>> 0 tcp -- last acquired _at_ order list:0
>>> 1  tcpinp -- last acquired _at_ order list:0
>>> 3   so_snd -- (already displayed)
>>> 0 udp -- last acquired _at_ order list:0
>>> 1  udpinp -- last acquired _at_ order list:0
>>> 3   so_snd -- (already displayed)
>>> 0 unp -- last acquired _at_ order list:0
>>> 3  so_snd -- (already displayed)
>>> 0 in_multi_mtx -- last acquired _at_ order list:0
>>> 1  igmp_mtx -- last acquired _at_ order list:0
>>> 2   if_addr_mtx -- last acquired _at_ order list:0
>>> 0 Giant -- last acquired _at_ /usr/src/sys/kern/subr_witness.c:513
>>> 1  eventhandler list -- last acquired _at_
>>> /usr/src/sys/kern/subr_eventhandler.c:132 1  vm object_list -- last
>>> acquired _at_ /usr/src/sys/vm/vm_object.c:229 1  system map -- (already
>>> displayed)
>>> 1  kobj -- last acquired _at_ /usr/src/sys/kern/subr_kobj.c:148
>>> 1  eventhandler -- last acquired _at_
>>> /usr/src/sys/kern/subr_eventhandler.c:82 1  filedesc structure -- last
>>> acquired _at_ /usr/src/sys/kern/kern_descrip.c:1512 2   accept -- last
>>> acquired _at_ order list:0
>>> 3    so_snd -- (already displayed)
>>> 2   pipe mutex -- last acquired _at_ order list:0
>>> 3    sigio lock -- last acquired _at_ order list:0
>>> 4     process group -- last acquired _at_ /usr/src/sys/kern/kern_fork.c:562
>>> 5      process lock -- last acquired _at_
>>> /usr/src/sys/kern/kern_kthread.c:96 6	ktrace -- last acquired _at_
>>> /usr/src/sys/kern/kern_fork.c:590
>>> 6	sigacts -- last acquired _at_ /usr/src/sys/kern/kern_kthread.c:98
>>> 6	session -- last acquired _at_ /usr/src/sys/kern/kern_fork.c:572
>>> 7	 uidinfo hash -- last acquired _at_ /usr/src/sys/kern/kern_resource.c:1032
>>> 8	  sleep mtxpool -- last acquired _at_
>>> /usr/src/sys/kern/kern_resource.c:914 8	  uidinfo struct -- last acquired
>>> _at_ order list:0
>>> 9	   allprison -- last acquired _at_ order list:0
>>> 1  ithread -- last acquired _at_ /usr/src/sys/kern/kern_intr.c:275
>>> 1  malloc -- last acquired _at_ /usr/src/sys/kern/kern_malloc.c:598
>>> 1  kernel linker -- last acquired _at_ /usr/src/sys/kern/kern_linker.c:470
>>> 1  ACPI semaphore -- last acquired _at_
>>> /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/Osd/OsdSynch.c:301 1
>>> TID lock -- last acquired _at_ /usr/src/sys/kern/subr_unit.c:592 1  standard
>>> object -- last acquired _at_ /usr/src/sys/vm/vm_glue.c:290 1  ttylist --
>>> last acquired _at_ /usr/src/sys/kern/tty.c:2865
>>> 1  GEOM orphanage -- last acquired _at_ /usr/src/sys/geom/geom_event.c:300
>>> 0 proctree -- last acquired _at_ /usr/src/sys/kern/kern_fork.c:561
>>> 4  process group -- (already displayed)
>>> 1  allproc -- last acquired _at_ /usr/src/sys/kern/kern_fork.c:286
>>> 5   process lock -- (already displayed)
>>> Spin locks:
>>> 0 ap boot -- last acquired _at_ order list:0
>>> 1  rm.mutex_mtx -- last acquired _at_ order list:0
>>> 2   sio -- last acquired _at_ order list:0
>>> 3    cy -- last acquired _at_ order list:0
>>> 4     uart_hwmtx -- last acquired _at_ order list:0
>>> 5      sabtty -- last acquired _at_ order list:0
>>> 6	zstty -- last acquired _at_ order list:0
>>> 7	 ng_node -- last acquired _at_ order list:0
>>> 8	  ng_worklist -- last acquired _at_ order list:0
>>> 9	   taskqueue_fast -- last acquired _at_ order list:0
>>> 10	    intr table -- last acquired _at_
>>> /usr/src/sys/i386/i386/intr_machdep.c:89 11	     ithread table lock --
>>> last acquired _at_ order list:0
>>> 12	      sleepq chain -- last acquired _at_
>>> /usr/src/sys/kern/subr_sleepqueue.c:218 13	       sched lock -- last
>>> acquired _at_ /usr/src/sys/kern/kern_kthread.c:116 14		turnstile chain --
>>> last acquired _at_ order list:0
>>> 15		 td_contested -- last acquired _at_ order list:0
>>> 16		  callout -- last acquired _at_ /usr/src/sys/kern/kern_timeout.c:439
>>> 17		   entropy harvest mutex -- last acquired _at_ order list:0
>>> 18		    allpmaps -- last acquired _at_ /usr/src/sys/i386/i386/pmap.c:1429
>>> 19		     vm page queue free mutex -- last acquired _at_
>>> /usr/src/sys/vm/vm_page.c:787 20		      icu -- last acquired _at_ order
>>> list:0
>>> 21		       smp rendezvous -- last acquired _at_ order list:0
>>> 22			tlb -- last acquired _at_ order list:0
>>> 23			 clk -- last acquired _at_ /usr/src/sys/i386/isa/clock.c:260
>>> 24			  mutex profiling lock -- last acquired _at_ order list:0
>>> 25			   kse zombie lock -- last acquired _at_ order list:0
>>> 26			    ALD Queue -- last acquired _at_ order list:0
>>> 27			     pcicfg -- last acquired _at_ order list:0
>>> 28			      NDIS thread lock -- last acquired _at_ order list:0
>>> 29			       tw_osl_io_lock -- last acquired _at_ order list:0
>>> 30				tw_osl_q_lock -- last acquired _at_ order list:0
>>> 31				 tw_cl_io_lock -- last acquired _at_ order list:0
>>> 32				  tw_cl_intr_lock -- last acquired _at_ order list:0
>>> 33				   tw_cl_gen_lock -- last acquired _at_ order list:0
>>> Locks which were never acquired:
>>> LED sx
>>> LED mtx
>>> tty
>>> nfslock
>>> random reseed
>>> devd
>>> bio queue
>>> GEOM topology
>>> if_cloners lock
>>> ifnet
>>> acpica subsystem lock
>>> ACPI global lock
>>> sf_buf
>>> taskqueue list
>>> uma object
>>> p_peers
>>> struct pargs.ref
>>> pbuf mutex
>>> bdone lock
>>> buffer daemon lock
>>> needsbuffer lock
>>> runningbufspace lock
>>> buf queue lock
>>> callout_wait_lock
>>> ACPI embedded controller
>>> ACPI CPU
>>> ACPI PCI power methods
>>> ACPI cmbat
>>> ACPI generic battery
>>> ACPI AC adapter
>>> ACPI thermal zone
>>> ACPI PCI link
>>> ACPI root bus
>>> ACPI PCI bus methods
>>> ACPI lid
>>> ACPI power resources
>>> umtxq_lock
>>> bounce pages lock
>>> arc4_mtx
>>> securelevel mutex lock
>>> accounting
>>> knlist lock for lockless objects
>>> net80211 instances
>>> rtsock route_cb lock
>>> Softdep Lock
>>> domain list
>>> rawcb
>>> pfil_head_list lock
>>> mountlist
>>> pmc shared lock
>>> accept_filter_mtx
>>> protect sysfilt_ops
>>> kqueue order
>>> UUID generator mutex lock
>>> unit# allocation
>>> Name Cache
>>> fifo mutex
>>> so_glabel
>>> encapmtx
>>> DEVFS ruleset lock
>>> fdesc
>>> filelist lock
>>> sysctl lock
>>> phys_pager list
>>> dev_pager list
>>> dev_pager create
>>> swapdev
>>> swap_pager list
>>> vm map sleep mutex
>>> PMAP2
>>> vm86 lock
>>> db> cont
>>> Uptime: 1s
>>> Automatic reboot in 15 seconds - press a key on the console to abort
>>> Rebooting...
>>>
>>>
>>> And, attempts at adding:
>>> options INVARIANTS
>>>
>>> failed to compile with the following:
>>>
>>> /usr/src/sys/dev/ata/atapi-cd.c: In function `acd_geom_attach':
>>> /usr/src/sys/dev/ata/atapi-cd.c:181: warning: implicit declaration of
>>> function `_sx_assert' /usr/src/sys/dev/ata/atapi-cd.c:181: warning:
>>> nested extern declaration of `_sx_assert'
>>>
>>>
>>>
>>> Since I am having a hang that feels the same to me in both RELENG_5 and
>>> RELENG_6, that's the main thing I would like to look at.  I am not very
>>> conversant at what I will need for kernel debugging (knowing which
>>> addresses to dump or structures to inspect) to debug this myself, but
>>> if you can tell me any further commands to issue in DDB or which
>>> addresses would assist in diagnosing or config options to add, that will
>>> be no problem and I will be more than happy to assist in any way!
>>>
>>>
>>> Thank you,
>>> Philip
>>> _______________________________________________
>>> freebsd-current_at_freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current
>>> To unsubscribe, send any mail to
>>> "freebsd-current-unsubscribe_at_freebsd.org"
>>
>> _______________________________________________
>> freebsd-current_at_freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>
> -- 
> John Baldwin <jhb_at_FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
> "Power Users Use the Power to Serve"  =  http://www.FreeBSD.org
>
Received on Wed Oct 19 2005 - 15:49:33 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:38:45 UTC