Re: crash with cpucontrol/microcode update : Today's -CURRENT

From: Larry Rosenman <ler_at_lerctr.org>
Date: Sat, 10 Aug 2013 14:17:40 -0500
On 2013-08-10 14:06, Larry Rosenman wrote:
> I'm getting the following _at_R254183:
> when I try to run the microcode_update.
> 
> Just started with yesterday's -CURRENT.
> 
> I can supply the full vmcore and ssh access.
> 
> 
> borg.lerctr.org dumped core - see /var/crash/vmcore.7
> 
> Sat Aug 10 14:00:00 CDT 2013
> 
> FreeBSD borg.lerctr.org 10.0-CURRENT FreeBSD 10.0-CURRENT #16: Sat Aug
> 10 13:45:06 CDT 2013
> root_at_borg.lerctr.org:/usr/obj/usr/src/sys/BORG-DTRACE  amd64
> 
> panic: page fault
> 
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and 
> you are
> welcome to change it and/or distribute copies of it under certain 
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for 
> details.
> This GDB was configured as "amd64-marcel-freebsd"...
> 
> Unread portion of the kernel message buffer:
> 			= DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags	= interrupt enabled, resume, IOPL = 0
> current process		= 1378 (cpucontrol)
> trap number		= 12
> panic: page fault
> cpuid = 0
> Uptime: 1m48s
> Dumping 2337 out of 64479 
> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
> 
> Reading symbols from /boot/kernel/zfs.ko.symbols...done.
> Loaded symbols for /boot/kernel/zfs.ko.symbols
> Reading symbols from /boot/kernel/acl_nfs4.ko.symbols...done.
> Loaded symbols for /boot/kernel/acl_nfs4.ko.symbols
> Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
> Loaded symbols for /boot/kernel/opensolaris.ko.symbols
> Reading symbols from /boot/kernel/linux.ko.symbols...done.
> Loaded symbols for /boot/kernel/linux.ko.symbols
> Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done.
> Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols
> Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done.
> Loaded symbols for /boot/kernel/snd_spicds.ko.symbols
> Reading symbols from /boot/kernel/sound.ko.symbols...done.
> Loaded symbols for /boot/kernel/sound.ko.symbols
> Reading symbols from /boot/kernel/coretemp.ko.symbols...done.
> Loaded symbols for /boot/kernel/coretemp.ko.symbols
> Reading symbols from /boot/kernel/ichsmb.ko.symbols...done.
> Loaded symbols for /boot/kernel/ichsmb.ko.symbols
> Reading symbols from /boot/kernel/smbus.ko.symbols...done.
> Loaded symbols for /boot/kernel/smbus.ko.symbols
> Reading symbols from /boot/kernel/ichwd.ko.symbols...done.
> Loaded symbols for /boot/kernel/ichwd.ko.symbols
> Reading symbols from /boot/kernel/cpuctl.ko.symbols...done.
> Loaded symbols for /boot/kernel/cpuctl.ko.symbols
> Reading symbols from /boot/kernel/crypto.ko.symbols...done.
> Loaded symbols for /boot/kernel/crypto.ko.symbols
> Reading symbols from /boot/kernel/cryptodev.ko.symbols...done.
> Loaded symbols for /boot/kernel/cryptodev.ko.symbols
> Reading symbols from /boot/kernel/dtraceall.ko.symbols...done.
> Loaded symbols for /boot/kernel/dtraceall.ko.symbols
> Reading symbols from /boot/kernel/profile.ko.symbols...done.
> Loaded symbols for /boot/kernel/profile.ko.symbols
> Reading symbols from /boot/kernel/cyclic.ko.symbols...done.
> Loaded symbols for /boot/kernel/cyclic.ko.symbols
> Reading symbols from /boot/kernel/dtrace.ko.symbols...done.
> Loaded symbols for /boot/kernel/dtrace.ko.symbols
> Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done.
> Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols
> Reading symbols from /boot/kernel/systrace.ko.symbols...done.
> Loaded symbols for /boot/kernel/systrace.ko.symbols
> Reading symbols from /boot/kernel/sdt.ko.symbols...done.
> Loaded symbols for /boot/kernel/sdt.ko.symbols
> Reading symbols from /boot/kernel/lockstat.ko.symbols...done.
> Loaded symbols for /boot/kernel/lockstat.ko.symbols
> Reading symbols from /boot/kernel/fasttrap.ko.symbols...done.
> Loaded symbols for /boot/kernel/fasttrap.ko.symbols
> Reading symbols from /boot/kernel/fbt.ko.symbols...done.
> Loaded symbols for /boot/kernel/fbt.ko.symbols
> Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done.
> Loaded symbols for /boot/kernel/dtnfscl.ko.symbols
> Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done.
> Loaded symbols for /boot/kernel/dtmalloc.ko.symbols
> Reading symbols from /boot/kernel/dtio.ko.symbols...done.
> Loaded symbols for /boot/kernel/dtio.ko.symbols
> Reading symbols from /boot/modules/vboxdrv.ko...done.
> Loaded symbols for /boot/modules/vboxdrv.ko
> Reading symbols from /boot/modules/nvidia.ko...done.
> Loaded symbols for /boot/modules/nvidia.ko
> Reading symbols from /boot/kernel/fdescfs.ko.symbols...done.
> Loaded symbols for /boot/kernel/fdescfs.ko.symbols
> Reading symbols from /boot/kernel/uhid.ko.symbols...done.
> Loaded symbols for /boot/kernel/uhid.ko.symbols
> Reading symbols from /boot/kernel/ums.ko.symbols...done.
> Loaded symbols for /boot/kernel/ums.ko.symbols
> Reading symbols from /boot/modules/vboxnetflt.ko...done.
> Loaded symbols for /boot/modules/vboxnetflt.ko
> Reading symbols from /boot/kernel/netgraph.ko.symbols...done.
> Loaded symbols for /boot/kernel/netgraph.ko.symbols
> Reading symbols from /boot/kernel/ng_ether.ko.symbols...done.
> Loaded symbols for /boot/kernel/ng_ether.ko.symbols
> Reading symbols from /boot/modules/vboxnetadp.ko...done.
> Loaded symbols for /boot/modules/vboxnetadp.ko
> #0  doadump (textdump=<value optimized out>) at pcpu.h:236
> 236	pcpu.h: No such file or directory.
> 	in pcpu.h
> (kgdb) #0  doadump (textdump=<value optimized out>) at pcpu.h:236
> #1  0xffffffff8051d6f0 in kern_reboot (howto=260)
>     at /usr/src/sys/kern/kern_shutdown.c:447
> #2  0xffffffff8051da77 in panic (fmt=<value optimized out>)
>     at /usr/src/sys/kern/kern_shutdown.c:754
> #3  0xffffffff80780d9a in trap_fatal (frame=<value optimized out>,
>     eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:873
> #4  0xffffffff80781199 in trap_pfault (frame=0x0, usermode=0)
>     at /usr/src/sys/amd64/amd64/trap.c:731
> #5  0xffffffff80780792 in trap (frame=0xffffff900d55c7b0)
>     at /usr/src/sys/amd64/amd64/trap.c:463
> #6  0xffffffff8076ad02 in calltrap ()
>     at /usr/src/sys/amd64/amd64/exception.S:232
> #7  0xffffffff80709be9 in vm_page_unwire (m=0x0, activate=0)
>     at /usr/src/sys/vm/vm_page.c:2356
> #8  0xffffffff806fb4ed in kmem_unback (object=0xffffffff80c57550,
>     addr=<value optimized out>, size=4096) at 
> /usr/src/sys/vm/vm_kern.c:404
> #9  0xffffffff806fb5a4 in kmem_free (vmem=0xffffffff80bd7780,
>     addr=18446741884987129872, size=4096) at 
> /usr/src/sys/vm/vm_kern.c:421
> #10 0xffffffff80506597 in contigfree (addr=0x0, size=4048,
>     type=0xffffffff812d5ea0) at /usr/src/sys/kern/kern_malloc.c:435
> #11 0xffffffff812d5a79 in cpuctl_ioctl (dev=<value optimized out>,
>     cmd=<value optimized out>, data=0xfffffe000d2a2f80 "0?c",
>     flags=<value optimized out>, td=<value optimized out>)
>     at /usr/src/sys/modules/cpuctl/../../dev/cpuctl/cpuctl.c:480
> #12 0xffffffff8041962f in devfs_ioctl_f (fp=0xfffffe001e68cbe0,
>     com=3222299396, data=0xfffffe000d2a2f80, cred=<value optimized 
> out>,
>     td=0xfffffe0252ebd490) at /usr/src/sys/fs/devfs/devfs_vnops.c:757
> #13 0xffffffff8056b3be in kern_ioctl (td=0xfffffe0252ebd490,
>     fd=<value optimized out>, com=0) at file.h:306
> #14 0xffffffff8056b13f in sys_ioctl (td=0xfffffe0252ebd490,
>     uap=0xffffff900d55cb80) at /usr/src/sys/kern/sys_generic.c:693
> #15 0xffffffff80781697 in amd64_syscall (td=0xfffffe0252ebd490, 
> traced=0)
>     at subr_syscall.c:134
> #16 0xffffffff8076afeb in Xfast_syscall ()
>     at /usr/src/sys/amd64/amd64/exception.S:391
> #17 0x000000080094b4ea in ?? ()
> Previous frame inner to this frame (corrupt stack?)
> Current language:  auto; currently minimal
> (kgdb)
> 
> ------------------------------------------------------------------------
> ps -axl
> 
>  UID  PID PPID CPU PRI NI    VSZ  RSS MWCHAN   STAT TT     TIME COMMAND
>    0    0    0   0  -8  0      0    0 -        DLs   -  0:02.61 
> [kernel]
>    0    1    0   0  52  0   9428    0 wait     DLs   -  0:00.26 [init]
>    0    2    0   0 -16  0      0    0 ftcl     DL    -  0:00.00 
> [ftcleanup]
>    0    3    0   0 -16  0      0    0 crypto_w DL    -  0:00.00 
> [crypto]
>    0    4    0   0 -16  0      0    0 crypto_r DL    -  0:00.00 [crypto 
> returns
>    0    5    0   0 -16  0      0    0 -        DL    -  0:00.00 [fdc0]
>    0    6    0   0  -8  0      0    0 tx->tx_s DL    -  0:00.09 
> [zfskern]
>    0    7    0   0 -16  0      0    0 waiting_ DL    -  0:00.00 
> [sctp_iterator]
>    0    8    0   0 -16  0      0    0 ccb_scan DL    -  0:00.00 
> [xpt_thrd]
>    0    9    0   0 -16  0      0    0 psleep   DL    -  0:00.00 
> [pagedaemon]
>    0   10    0   0 -16  0      0    0 audit_wo DL    -  0:00.00 [audit]
>    0   11    0   0 155  0      0    0 -        RL    -  6:08.99 [idle]
>    0   12    0   0 -84  0      0    0 -        WL    -  0:00.43 [intr]
>    0   13    0   0  -8  0      0    0 -        DL    -  0:00.38 [geom]
>    0   14    0   0 -16  0      0    0 -        DL    -  0:00.01 
> [yarrow]
>    0   15    0   0 -68  0      0    0 -        DL    -  0:00.04 [usb]
>    0   16    0   0 -20  0      0    0 VBoxIS   DL    -  0:00.00 [TIMER]
>    0   17    0   0 -16  0      0    0 psleep   DL    -  0:00.00 
> [vmdaemon]
>    0   18    0   0 155  0      0    0 pgzero   DL    -  0:00.00 
> [pagezero]
>    0   19    0   0 -16  0      0    0 psleep   DL    -  0:00.00 
> [bufdaemon]
>    0   20    0   0  16  0      0    0 syncer   DL    -  0:00.00 
> [syncer]
>    0   21    0   0 -16  0      0    0 vlruwt   DL    -  0:00.00 [vnlru]
>    0  847    1   0  52  0  16584    0 select   Ds    -  0:00.00 
> [moused]
>    0  866    1   0  20  0  13208    0 select   Ds    -  0:00.00 [devd]
>    0 1004    1   0  20  0  14380    0 -        Rs    -  0:00.05 
> [syslogd]
>    0 1007    1   0 -52  0   6192 2104 nanslp   Ds    -  0:00.00 
> [watchdogd]
>    0 1021    1   0  20  0  16468    0 select   Ds    -  0:00.01 
> [rpcbind]
>    0 1056    1   0  52  0  38964    0 select   Ds    -  0:00.02 
> [mountd]
>    0 1062    1   0  52  0  36804    0 select   Ds    -  0:00.04 [nfsd]
>    0 1064 1062   0  52  0  12228    0 rpcsvc   D     -  0:00.00 [nfsd]
>    0 1072    1   0  52  0  32716    0 select   Ds    -  0:00.99 
> [apcupsd]
>    0 1103    1   0  20  0  25268    0 select   Ds    -  0:00.01 [ntpd]
>    0 1124    0   0 -16  0      0    0 sleep    DL    -  0:00.00 
> [ng_queue]
>    0 1136    1   0  20  0  34748    0 nanslp   Ds    -  0:00.01 [perl]
>    0 1140    1   0  52  0  30844    0 nanslp   D     -  0:00.43 
> [smartd]
>   70 1149    1   0  20  0  86688    0 select   Ds    -  0:00.23 
> [postgres]
>   70 1153 1149   0  20  0  86688    0 select   Ds    -  0:00.00 
> [postgres]
>   70 1154 1149   0  20  0  86688    0 select   Ds    -  0:00.00 
> [postgres]
>   70 1155 1149   0  20  0  86688    0 select   Ds    -  0:00.00 
> [postgres]
>   70 1156 1149   0  20  0  46392    0 select   Ds    -  0:00.00 
> [postgres]
>   26 1246    1   0  52  0  48804    0 select   Ds    -  0:00.00 
> [exim-4.80.1-2]
>    0 1255    1   0  20  0 103300    0 kqread   Ds    -  0:00.02 [cupsd]
> 1028 1261    1   0 155  0  46764    0 select   Ds    -  0:00.07 
> [boinc_client]
>  910 1265    1   0  20  0  65436    0 uwait    Ds    -  0:00.00 
> [bacula-sd]
>    0 1268    1   0  20  0  65260    0 uwait    Ds    -  0:00.00 
> [bacula-fd]
>  910 1271    1   0  20  0  78464    0 uwait    Ds    -  0:00.00 
> [bacula-dir]
>    0 1276    1   0  20  0  56324    0 select   Ds    -  0:00.00 [sshd]
>    0 1288    1   0  20  0  16472    0 nanslp   Ds    -  0:00.00 [cron]
>    0 1310    1   0  52  0  18592    0 select   Ds    -  0:00.00 [inetd]
>    0 1326    1   0  52  0  16944    0 wait     D     -  0:00.00 [sh]
>    0 1327    1   0  52  0  12220    0 piperd   D     -  0:00.00 
> [logger]
>    0 1328 1326   0  52  0   8120    0 nanslp   D     -  0:00.00 [sleep]
>    0 1330    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
>    0 1331    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
>    0 1332    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
>    0 1333    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
>    0 1334    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
>    0 1335    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
>    0 1336    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.00 [getty]
>    0 1337    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
>    0 1339 1276   0  21  0  81592    0 select   Ds    -  0:00.00 [sshd]
> 1002 1341 1339   0  20  0  81592    0 select   D     -  0:00.00 [sshd]
> 1002 1342 1341   0  20  0  16944    0 wait     Ds    -  0:00.00 [sh]
>    0 1344 1342   0  20  0  51140    0 select   D     -  0:00.00 [sudo]
>    0 1345 1344   0  20  0  16944    0 wait     D     -  0:00.00 [sh]
> 1028 1346 1261   0 155 19  86380    0 -        RN    -  0:00.94 
> [wcgrid_faah_7.
> 1028 1347 1261   0 155 19   1892    0 -        RN    -  0:00.04 
> [wcgrid_cep2_6.
> 1028 1348 1261   0 155 19  68404    0 -        RN    -  0:01.31 
> [wcgrid_faah_7.
> 1028 1349 1261   0 155 19  68404    0 -        RN    -  0:00.74 
> [wcgrid_faah_7.
> 1028 1350 1261   0 155 19  90036    0 -        RN    -  0:00.90 
> [wcgrid_faah_7.
> 1028 1351 1261   0 155 19  68412    0 -        RN    -  0:00.73 
> [wcgrid_faah_7.
> 1028 1352 1261   0 155 19  67892    0 -        RN    -  0:00.88 
> [wcgrid_faah_7.
> 1028 1353 1347   0 155 19   1892    0 i        DN    -  0:00.00 
> [wcgrid_cep2_6.
> 1028 1354 1261   0 155 19  76092    0 -        RN    -  0:00.80 
> [wcgrid_faah_7.
> 1028 1355 1353   0 155 19   1892    0 i        DN    -  0:00.00 
> [wcgrid_cep2_6.
> 1028 1356 1261   0 155 19  68404    0 i        DN    -  0:00.00 
> [wcgrid_faah_7.
> 1028 1357 1261   0 155 19  68412    0 -        RN    -  0:00.00 
> [wcgrid_faah_7.
> 1028 1358 1261   0 155 19  86380    0 -        RN    -  0:00.00 
> [wcgrid_faah_7.
> 1028 1359 1261   0 155 19  68404    0 i        DN    -  0:00.00 
> [wcgrid_faah_7.
> 1028 1360 1261   0 155 19  67892    0 i        DN    -  0:00.00 
> [wcgrid_faah_7.
> 1028 1361 1261   0 155 19  76092    0 i        DN    -  0:00.00 
> [wcgrid_faah_7.
> 1028 1362 1261   0 155 19  90036    0 -        RN    -  0:00.00 
> [wcgrid_faah_7.
>    0 1372 1345   0  43  0  16944    0 wait     D+    -  0:00.01 [sh]
>    0 1378 1372   0  30  0  12228    0 -        R+    -  0:00.00 
> [cpucontrol]
> 
> ------------------------------------------------------------------------
> vmstat -s
> 
>    319667 cpu context switches
>     33520 device interrupts
>      4096 software interrupts
>    633273 traps
>   5813583 system calls
>        22 kernel threads created
>      1128  fork() calls
>       219 vfork() calls
>         9 rfork() calls
>         0 swap pager pageins
>         0 swap pager pages paged in
>         0 swap pager pageouts
>         0 swap pager pages paged out
>      1154 vnode pager pageins
>      8819 vnode pager pages paged in
>         8 vnode pager pageouts
>        16 vnode pager pages paged out
>         0 page daemon wakeups
>         0 pages examined by the page daemon
>        11 pages reactivated
>     42644 copy-on-write faults
>       255 copy-on-write optimized faults
>    562815 zero fill pages zeroed
>         0 zero fill pages prezeroed
>         5 intransit blocking page faults
>    622067 total VM faults taken
>      1014 page faults requiring I/O
>         0 pages affected by kernel thread creation
>     42834 pages affected by  fork()
>      8586 pages affected by vfork()
>    115675 pages affected by rfork()
>         0 pages cached
>    823364 pages freed
>         0 pages freed by daemon
>         0 pages freed by exiting processes
>     26936 pages active
>      8342 pages inactive
>         4 pages in VM cache
>    220798 pages wired down
>  15823471 pages free
>      4096 bytes per page
>    240707 total name lookups
>           cache hits (90% pos + 4% neg) system 0% per-directory
>           deletions 0%, falsehits 0%, toolong 0%
> 
> ------------------------------------------------------------------------
> vmstat -m
> 
>          Type InUse MemUse HighUse Requests  Size(s)
>       kdtrace   462   100K       -     1922  64,256
>          kenv    84    11K       -      112  16,32,64,128
>        kqueue     2     3K       -       30  256,2048
>     proc-args    53     5K       -      513  16,32,64,128,256
>        DEVFSP     3     1K       -       38  64
>         hhook     2     1K       -        2  256
>       ithread   122    22K       -      122  32,128,256
>        KTRACE   100    13K       -      100  128
>        linker   401   197K       -      544  
> 16,32,64,128,256,512,1024,2048,4096
>         lockf    53     6K       -       89  64,128
>    loginclass     1     1K       -        6  64
>         cache     1     1K       -        1  32
>        devbuf 18685 36135K       -    19791  
> 16,32,64,128,256,512,1024,2048,4096
>          temp    57    35K       -     6961  
> 16,32,64,128,256,512,1024,2048,4096
>        ip6ndp     5     1K       -        6  64,128
>        module   301    38K       -      301  128
>      mtx_pool     2    16K       -        2
>           osd     7     1K       -       18  16,32,64,128
>      pmchooks     1     1K       -        1  128
>        NFS fh     1     1K       -        5  16
>          pgrp    40     5K       -       62  128
>       session    37     5K       -       48  128
>          proc     2   256K       -        2
>       subproc   194   360K       -     1489  512,4096
>          cred   108    17K       -     8666  64,256
>        plimit    23     6K       -      204  256
>       uidinfo     7    33K       -       25  128
>        sysctl     0     0K       -      526  16,32,64
>     sysctloid  6788   337K       -     6940  16,32,64,128
>     sysctltmp     0     0K       -      384  16,32,64,128,256,4096
>       tidhash     1   256K       -        1
>       callout     9  3208K       -        9
>          umtx   792    99K       -      792  128
>      p1003.1b     1     1K       -        1  16
>          SWAP    12 19681K       -       12  64
>           bus   945    90K       -     5370  16,32,64,128,256,512,1024
>        bus-sc   141   293K       -     2331  
> 16,32,64,128,256,512,1024,2048,4096
>       devstat    34    69K       -       34  32,4096
>  eventhandler    90     8K       -       90  64,128
>          kobj   180   720K       -      902  4096
>       Per-cpu     1     1K       -        1  32
>          rman   360    41K       -      797  16,32,128
>          sbuf     0     0K       -     3645  
> 16,32,64,128,256,512,1024,2048,4096
>        sglist     3     1K       -        3  32
>         stack     0     0K       -        2  256
>     taskqueue   109    17K       -      139  16,32,64,256,1024
>        Unitno    20     2K       -      420  32,64
>          vmem     2   160K       -        4
>      ioctlops     1     1K       -     2920  
> 16,32,64,128,256,512,1024,2048
>        select    35     5K       -       35  128
>           iov     1     1K       -     1252  16,64,128,256,512
>           msg     4    30K       -        4  2048,4096
>           sem     4   106K       -        4  2048,4096
>           shm     6    30K       -       13  2048
>           tty    20    20K       -       22  1024,2048
>           pts     1     1K       -        1  256
>      mbuf_tag     0     0K       -       59  32,128
>         shmfd     1     8K       -        1
>        soname     7     1K       -    22000  16,32,128
>           pcb    37  8341K       -       77  16,32,128,1024,2048
>     newnfsmnt     1     1K       -        1  1024
>      vfscache     1 16384K       -        1
>      vfs_hash     1  8192K       -        1
>        vnodes     1     1K       -        1  256
>     pfs_nodes    21     6K       -       21  256
>   pfs_vncache    86     6K       -       88  64
>         mount   396    20K       -     1854  16,32,64,128,256,512
>          GEOM   344    60K       -     3318  
> 16,32,64,128,256,512,1024,2048
>   vnodemarker     0     0K       -      161  512
>           BPF     3     1K       -        3  128
>        acpica  1830   187K       -    52091  
> 16,32,64,128,256,512,1024,2048
>         ifnet     4     7K       -        4  128,2048
>        ifaddr    48    15K       -       48  
> 32,64,128,256,512,2048,4096
>   ether_multi    47     3K       -       54  16,32,64
>         clone     6     1K       -        6  128
>        arpcom     2     1K       -        2  16
>       lltable    13     5K       -       13  256,512
>          UART     6     5K       -        6  16,1024
>       CAM CCB    13    26K       -      101  2048
>      acpitask     1     8K       -        1
>       acpisem    22     3K       -       22  128
>      CAM path    22     1K       -       68  32
>       acpidev    36     3K       -       36  64
>      routetbl    35     5K       -      228  32,64,128,256,512
>          igmp     3     1K       -        3  256
>      in_multi     2     1K       -        2  256
>    CAM periph    16     4K       -       40  16,32,64,128,256
>     CAM queue    31     8K       -      100  16,32,512
> CAM dev queue     8     1K       -        8  32
>     sctp_a_it     0     0K       -        3  16
>      sctp_vrf     1     1K       -        1  64
>      sctp_ifa     5     1K       -        5  128
>      sctp_ifn     2     1K       -        2  128
>     sctp_iter     0     0K       -        3  256
>     hostcache     1    28K       -        1
>      syncache     1    64K       -        1
>   in6_mfilter     1     1K       -        1  1024
>     in6_multi    27     4K       -       27  32,256
>  ip6_moptions     2     1K       -        2  32,256
>     raid_data     0     0K       -      480  32,128,256
>           mld     3     1K       -        3  128
>       CAM SIM     8     2K       -        8  256
>       CAM XPT    58     4K       -      342  16,32,64,128,256,1024
>       NFS FHA     1     2K       -        1  2048
>           rpc    19     5K       -       23  32,64,128,256,512,1024
> audit_evclass   188     6K       -      229  32
>     vm_pgdata     7  8193K       -        7  128
>       UMAHash     3     5K       -        6  512,1024,2048,4096
>       scsi_cd     0     0K       -       11  16
>           USB    50   162K       -       56  
> 16,32,64,128,256,512,1024,4096
>        USBdev    76    26K       -      112  32,64,128,512,2048,4096
>      pci_link    16     2K       -       16  32,64,128
>     acpi_perf     8     1K       -        8  64
>       memdesc     1     4K       -        1  4096
>      atkbddev     2     1K       -        2  64
> md_nvidia_data     0     0K       -       78  512
>        kbdmux     7    18K       -        7  16,512,1024,2048
>           LED     4     1K       -        4  16,128
>   md_sii_data     0     0K       -       78  512
>       CAM DEV    15    30K       -       24  2048
>       ata_pci     1     1K       -        1  64
>      acpiintr     1     1K       -        1  64
>      ppbusdev     2     1K       -        2  256
>        apmdev     1     1K       -        1  128
>    madt_table     0     0K       -        1  4096
>        isadev     5     1K       -        5  128
> random_adaptors     1     1K       -        1  32
>       entropy  1024    64K       -     1024  64
>        DEVFS3   223    56K       -      269  256
>       io_apic     2     4K       -        2  2048
>        DEVFS1   194    97K       -      236  512
>           MCA     8     1K       -        8  128
>          cdev     8     2K       -        8  256
>           msi     2     1K       -        2  128
>      nexusdev     4     1K       -        4  16
>         DEVFS    42     1K       -       43  16,128
>      filedesc    86   267K       -     1390  16,32,64,128,2048,4096
> filedesc_to_leader    17     2K       -       17  64
>         sigio     1     1K       -        1  64
>      filecaps     0     0K       -        2  64
>         linux    43     3K       -       43  32,64
>         mixer     1     4K       -        1  4096
>        feeder    16     2K       -       18  32,128
>    kstat_data     5     1K       -        5  64
>       solaris 64977 432023K       -   918878
> 16,32,64,128,256,512,1024,2048,4096
>        spicds     7     1K       -        7  128
>      envy24ht    15   195K       -       15  64,2048
>        cpuctl     2     5K       -        2  64,4096
>        crypto     1     1K       -        1  512
>         xform     0     0K       -      167  16,32
>        cyclic    32     3K       -       32  16,64,128
>           fbt     1   256K       -        1
>      iprtheap    23    56K       -       23  32,64,128,256,2048
>        nvidia   179   862K       -      182  
> 16,32,64,128,256,512,1024,2048,4096
>   fdesc_mount     1     1K       -        1  16
> netgraph_node     4     1K       -        4  128,256
>      netgraph     2     1K       -        2  64
> 
> ------------------------------------------------------------------------
> vmstat -z
> 
> ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL 
> SLEEP
> 
> UMA Kegs:               384,      0,     172,       8,     172,   0,   
> 0
> UMA Zones:             1664,      0,     188,       0,     188,   0,   
> 0
> UMA Slabs:               80,      0,   22943,     257,   39931,   0,   
> 0
> UMA RCntSlabs:           88,      0,     816,      39,     816,   0,   
> 0
> UMA Hash:               256,      0,       7,       8,      10,   0,   
> 0
> 4 Bucket:                32,      0,     252,     748,    5062, 786,   
> 0
> 8 Bucket:                64,      0,     134,    1230,    2869, 188,   
> 0
> 16 Bucket:              128,      0,     121,    1150,    2651,  26,   
> 0
> 32 Bucket:              256,      0,     169,     761,    4898,  92,   
> 0
> 64 Bucket:              512,      0,     427,     253,    3989, 334,   
> 0
> 128 Bucket:            1024,      0,    1577,     147,    6403,  23,   
> 0
> vmem btag:               56,      0,   15370,     605,   16546, 226,   
> 0
> VM OBJECT:              256,      0,    3138,     342,   18576,   0,   
> 0
> RADIX NODE:             144,      0,   13651,    1037,   93009,  39,   
> 0
> MAP:                    240,      0,       3,      61,       3,   0,   
> 0
> KMAP ENTRY:             128,      0,      13,     514,      13,   2,   
> 0
> MAP ENTRY:              128,      0,    1705,    1178,   41317,   0,   
> 0
> VMSPACE:                408,      0,      53,     181,    1357,   0,   
> 0
> fakepg:                 104,      0,       0,       0,       0,   0,   
> 0
> mt_zone:               4112,      0,     288,       0,     288,   0,   
> 0
> 16:                      16,      0,       5,     246,       5,   0,   
> 0
> 16:                      16,      0,    2467,     545,    2622,   0,   
> 0
> 16:                      16,      0,       2,     249,       2,   0,   
> 0
> 16:                      16,      0,      89,     915,   24074,   0,   
> 0
> 16:                      16,      0,       4,    1000,     254,   0,   
> 0
> 16:                      16,      0,     651,     855,    4300,   0,   
> 0
> 16:                      16,      0,      48,     454,      80,   0,   
> 0
> 16:                      16,      0,    3154,     862,   71341,   0,   
> 0
> 32:                      32,      0,       2,     123,       2,   0,   
> 0
> 32:                      32,      0,    2634,     491,    2720,   0,   
> 0
> 32:                      32,      0,      10,     240,      10,   0,   
> 0
> 32:                      32,      0,     109,     891,    4525,   0,   
> 0
> 32:                      32,      0,     250,     750,     649,   0,   
> 0
> 32:                      32,      0,     337,    1288,    2957,   0,   
> 0
> 32:                      32,      0,      79,     921,     480,   0,   
> 0
> 32:                      32,      0,    1963,    2412,   48008,   0,   
> 0
> 64:                      64,      0,       6,     428,       6,   0,   
> 0
> 64:                      64,      0,     330,     600,     340,   0,   
> 0
> 64:                      64,      0,      14,     296,      14,   0,   
> 0
> 64:                      64,      0,     781,    1017,    2205,   0,   
> 0
> 64:                      64,      0,     103,     951,    1404,   0,   
> 0
> 64:                      64,      0,    9185,     735,   16699,   0,   
> 0
> 64:                      64,      0,    1093,     829,    1093,   0,   
> 0
> 64:                      64,      0,   25104,    5524,  244655,   0,   
> 0
> 128:                    128,      0,       8,     519,       8,   0,   
> 0
> 128:                    128,      0,    2007,    1000,    2513,   0,   
> 0
> 128:                    128,      0,      45,     978,      67,   0,   
> 0
> 128:                    128,      0,    1239,     900,   22787,   0,   
> 0
> 128:                    128,      0,    1164,     603,    1245,   0,   
> 0
> 128:                    128,      0,    2042,     686,   25479,   0,   
> 0
> 128:                    128,      0,       3,     276,       3,   0,   
> 0
> 128:                    128,      0,    9616,    2288,   86497,   0,   
> 0
> 256:                    256,      0,       2,      73,       2,   0,   
> 0
> 256:                    256,      0,      34,     461,     241,   0,   
> 0
> 256:                    256,      0,       6,     129,       6,   0,   
> 0
> 256:                    256,      0,     242,     403,     902,   0,   
> 0
> 256:                    256,      0,     465,     300,     950,   0,   
> 0
> 256:                    256,      0,     397,     338,    5080,   0,   
> 0
> 256:                    256,      0,       1,      74,       1,   0,   
> 0
> 256:                    256,      0,    4397,    4423,   98046,   0,   
> 0
> 512:                    512,      0,       4,      84,       8,   0,   
> 0
> 512:                    512,      0,      39,      49,     195,   0,   
> 0
> 512:                    512,      0,       0,     232,     161,   0,   
> 0
> 512:                    512,      0,     235,     165,    1414,   0,   
> 0
> 512:                    512,      0,      18,      70,      18,   0,   
> 0
> 512:                    512,      0,     338,     126,     434,   0,   
> 0
> 512:                    512,      0,       0,       0,       0,   0,   
> 0
> 512:                    512,      0,   10447,     601,  209776,   0,   
> 0
> 1024:                  1024,      0,       1,      15,       1,   0,   
> 0
> 1024:                  1024,      0,       1,      67,     100,   0,   
> 0
> 1024:                  1024,      0,       5,      23,       5,   0,   
> 0
> 1024:                  1024,      0,       6,      62,    1720,   0,   
> 0
> 1024:                  1024,      0,      20,      20,      20,   0,   
> 0
> 1024:                  1024,      0,      26,     262,    3723,   0,   
> 0
> 1024:                  1024,      0,       1,      15,       1,   0,   
> 0
> 1024:                  1024,      0,     339,      93,   51688,   0,   
> 0
> 2048:                  2048,      0,       5,      31,      12,   0,   
> 0
> 2048:                  2048,      0,      32,      20,     129,   0,   
> 0
> 2048:                  2048,      0,       1,       5,       1,   0,   
> 0
> 2048:                  2048,      0,      15,      11,     227,   0,   
> 0
> 2048:                  2048,      0,      76,      32,    1373,   0,   
> 0
> 2048:                  2048,      0,      25,      33,     203,   0,   
> 0
> 2048:                  2048,      0,       5,       1,       5,   0,   
> 0
> 2048:                  2048,      0,     365,      45,    4185,   0,   
> 0
> 4096:                  4096,      0,       1,       0,       1,   0,   
> 0
> 4096:                  4096,      0,       2,       0,       2,   0,   
> 0
> 4096:                  4096,      0,     180,       1,     902,   0,   
> 0
> 4096:                  4096,      0,      12,       0,      14,   0,   
> 0
> 4096:                  4096,      0,       5,       2,       8,   0,   
> 0
> 4096:                  4096,      0,     132,       7,    4551,   0,   
> 0
> 4096:                  4096,      0,       0,       0,       0,   0,   
> 0
> 4096:                  4096,      0,    5874,    9408,   89410,   0,   
> 0
> uint64 pcpu:              8,      0,    1386,     150,    1386,   0,   
> 0
> SLEEPQUEUE:              80,      0,     397,     750,     397,   0,   
> 0
> Files:                   80,      0,     207,     793,   14296,   0,   
> 0
> TURNSTILE:              136,      0,     397,     283,     397,   0,   
> 0
> rl_entry:                40,      0,      49,     951,      49,   0,   
> 0
> umtx pi:                 96,      0,       0,       0,       0,   0,   
> 0
> MAC labels:              40,      0,       0,       0,       0,   0,   
> 0
> PROC:                  1208,      0,      83,      34,    1378,   0,   
> 0
> THREAD:                1168,      0,     377,      19,     542,   0,   
> 0
> cpuset:                  72,      0,     286,     759,     438,   0,   
> 0
> cyclic_id_cache:         64,      0,       0,       0,       0,   0,   
> 0
> audit_record:          1240,      0,       0,       0,       0,   0,   
> 0
> mbuf_packet:            256, 25727505,    1023,     607,    1407,   0,  
>  0
> mbuf:                   256, 25727505,       2,     878,    1060,   0,  
>  0
> mbuf_cluster:          2048, 4019922,    1625,       1,    1625,1490,   
> 0
> mbuf_jumbo_page:       4096, 2009961,       0,       3,       7,   0,   
> 0
> mbuf_jumbo_9k:         9216, 1786632,       0,       0,       0,   0,   
> 0
> mbuf_jumbo_16k:       16384, 1339972,       0,       0,       0,   0,   
> 0
> mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   
> 0
> dtrace_state_cache:    4096,      0,       0,       0,       0,   0,   
> 0
> ttyinq:                 160,      0,     180,     320,     540,   0,   
> 0
> ttyoutq:                256,      0,      95,     400,     287,   0,   
> 0
> g_bio:                  248,      0,       0,     736,  100826,   0,   
> 0
> ata_request:            336,      0,       0,     154,      70,   0,   
> 0
> vtnet_tx_hdr:            24,      0,       0,       0,       0,   0,   
> 0
> cryptop:                 88,      0,       0,       0,       0,   0,   
> 0
> cryptodesc:              72,      0,       0,       0,       0,   0,   
> 0
> nv_stack_t:           12288,      0,       2,       2,      12,   0,   
> 0
> FPU_save_area:          512,      0,       0,       0,       0,   0,   
> 0
> taskq_zone:              48,      0,       0,       0,       0,   0,   
> 0
> taskq_zone:              48,      0,       0,     581,      81,   0,   
> 0
> VNODE:                  472,      0,    9463,     113,    9663,   0,   
> 0
> VNODEPOLL:              112,      0,       0,       0,       0,   0,   
> 0
> BUF TRIE:               144,      0,       0,  105948,       0,   0,   
> 0
> NAMEI:                 1024,      0,       0,     132,   57320,   0,   
> 0
> S VFS Cache:            108,      0,    9378,     527,   11372,   0,   
> 0
> STS VFS Cache:          148,      0,       0,       0,       0,   0,   
> 0
> L VFS Cache:            328,      0,     151,     173,     205,   0,   
> 0
> LTS VFS Cache:          368,      0,       0,       0,       0,   0,   
> 0
> NCLNODE:                528,      0,       1,      13,       1,   0,   
> 0
> space_seg_cache:         64,      0,    7959,  101099,  238144,   0,   
> 0
> zio_cache:              944,      0,       6,   13970,  295331,   0,   
> 0
> zio_link_cache:          48,      0,       0,   15770,  278175,   0,   
> 0
> sa_cache:                80,      0,    9242,     658,    9413,   0,   
> 0
> dnode_t:                856,      0,   10211,      49,   13930,   0,   
> 0
> dmu_buf_impl_t:         224,      0,   19047,     299,   24850,   0,   
> 0
> arc_buf_hdr_t:          216,      0,   11737,     467,   13936,   0,   
> 0
> arc_buf_t:               72,      0,   10586,     689,   16271,   0,   
> 0
> zil_lwb_cache:          192,      0,       5,     495,      54,   0,   
> 0
> zfs_znode_cache:        368,      0,    9242,     128,    9413,   0,   
> 0
> Mountpoints:            816,      0,      30,      35,      30,   0,   
> 0
> pipe:                   744,      0,       9,      86,     814,   0,   
> 0
> ksiginfo:               112,      0,     131,    1374,    1687,   0,   
> 0
> itimer:                 352,      0,       1,      32,       1,   0,   
> 0
> KNOTE:                  128,      0,       6,     521,      40,   0,   
> 0
> socket:                 680, 2063352,      61,      71,    4016,   0,   
> 0
> unpcb:                  240, 2063360,      16,     480,    3676,   0,   
> 0
> ipq:                     56, 125670,       0,       0,       0,   0,   
> 0
> udp_inpcb:              392, 2063360,      20,     220,     298,   0,   
> 0
> udpcb:                   16, 2063471,      20,     984,     298,   0,   
> 0
> tcp_inpcb:              392, 2063360,      24,     216,      36,   0,   
> 0
> tcpcb:                 1024, 2063352,      24,      80,      36,   0,   
> 0
> tcptw:                   72,  27775,       0,     165,       2,   0,   
> 0
> syncache:               152,  15366,       0,      78,       1,   0,   
> 0
> hostcache:              136,  15370,       0,       0,       0,   0,   
> 0
> tcpreass:                40, 251300,       0,       0,       0,   0,   
> 0
> sackhole:                32,      0,       0,       0,       0,   0,   
> 0
> sctp_ep:               1408, 2063352,       0,       0,       0,   0,   
> 0
> sctp_asoc:             2344,  40000,       0,       0,       0,   0,   
> 0
> sctp_laddr:              48,  80012,       0,     332,       4,   0,   
> 0
> sctp_raddr:             728,  80000,       0,       0,       0,   0,   
> 0
> sctp_chunk:             136, 400026,       0,       0,       0,   0,   
> 0
> sctp_readq:             104, 400026,       0,       0,       0,   0,   
> 0
> sctp_stream_msg_out:    104, 400026,       0,       0,       0,   0,   
> 0
> sctp_asconf:             40, 400000,       0,       0,       0,   0,   
> 0
> sctp_asconf_ack:         48, 400060,       0,       0,       0,   0,   
> 0
> ripcb:                  392, 2063360,       0,       0,       0,   0,   
> 0
> rtentry:                200,      0,      17,     243,      17,   0,   
> 0
> selfd:                   56,      0,      76,     989,    4050,   0,   
> 0
> SWAPMETA:               288, 8039850,       0,       0,       0,   0,   
> 0
> NetGraph items:          72,   4123,       0,       0,       0,   0,   
> 0
> NetGraph data items:     72,    527,       0,       0,       0,   0,   
> 0
> 
> 
> ------------------------------------------------------------------------
> vmstat -i
> 
> interrupt                          total       rate
> irq1: atkbd0                           3          0
> irq6: fdc0                            22          0
> irq14: ata0                          117          0
> irq17: uhci0 ehci0                  1145          7
> irq19: uhci1 ahci0+                31666        206
> cpu0:timer                          7042         46
> irq256: em0                          567          3
> cpu6:timer                          5638         36
> cpu1:timer                          6302         41
> cpu3:timer                          7819         51
> cpu2:timer                          4971         32
> cpu7:timer                          6608         43
> cpu4:timer                          5406         35
> cpu5:timer                          6416         41
> Total                              83722        547
> 
> ------------------------------------------------------------------------
> pstat -T
> 
> 207/2063348 files
> 0M/147455M swap space
> 
> ------------------------------------------------------------------------
> pstat -s
> 
> Device          512-blocks     Used    Avail Capacity
> /dev/gpt/swap0    50331392        0 50331392     0%
> /dev/gpt/swap1    50331392        0 50331392     0%
> /dev/gpt/swap2    50331392        0 50331392     0%
> /dev/gpt/swap3    50331392        0 50331392     0%
> /dev/gpt/swap4    50331392        0 50331392     0%
> /dev/gpt/swap5    50331392        0 50331392     0%
> Total            301988352        0 301988352     0%
> 
> ------------------------------------------------------------------------
> iostat
> 
> iostat: kvm_read(_tk_nin): invalid address (0x0)
> iostat: disabling TTY statistics
>             ada0             ada1             ada2             cpu
>   KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
>  15.37  36  0.55  15.50  34  0.52  15.58  36  0.55   0  0  2  0 97
> 
> ------------------------------------------------------------------------
> ipcs -a
> 
> Message Queues:
> T           ID          KEY MODE        OWNER    GROUP    CREATOR
> CGROUP                 CBYTES                 QNUM
> QBYTES        LSPID        LRPID STIME    RTIME    CTIME
> 
> Shared Memory:
> T           ID          KEY MODE        OWNER    GROUP    CREATOR
> CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME
> DTIME    CTIME m        65536      5432001 --rw------- pgsql    pgsql
>   pgsql    pgsql               4     41263104         1149
> 1149 13:54:11 13:55:13 13:54:11
> 
> Semaphores:
> T           ID          KEY MODE        OWNER    GROUP    CREATOR
> CGROUP          NSEMS OTIME    CTIME s        65536      5432001
> --rw------- pgsql    pgsql    pgsql    pgsql              17 13:54:11
> 13:54:11
> s        65537      5432002 --rw------- pgsql    pgsql    pgsql
> pgsql              17 13:54:11 13:54:11
> s        65538      5432003 --rw------- pgsql    pgsql    pgsql
> pgsql              17 13:54:11 13:54:11
> s        65539      5432004 --rw------- pgsql    pgsql    pgsql
> pgsql              17 13:54:11 13:54:11
> s        65540      5432005 --rw------- pgsql    pgsql    pgsql
> pgsql              17 13:54:11 13:54:11
> s        65541      5432006 --rw------- pgsql    pgsql    pgsql
> pgsql              17 13:54:11 13:54:11
> s        65542      5432007 --rw------- pgsql    pgsql    pgsql
> pgsql              17 13:54:19 13:54:11
> 
> 
> ------------------------------------------------------------------------
> ipcs -T
> 
> msginfo:
> 	msgmax:        16384	(max characters in a message)
> 	msgmni:           40	(# of message queues)
> 	msgmnb:         2048	(max characters in a message queue)
> 	msgtql:           40	(max # of messages in system)
> 	msgssz:            8	(size of a message segment)
> 	msgseg:         2048	(# of message segments in system)
> 
> shminfo:
> 	shmmax:    536870912	(max shared memory segment size)
> 	shmmin:            1	(min shared memory segment size)
> 	shmmni:          192	(max number of shared memory identifiers)
> 	shmseg:          128	(max shared memory segments per process)
> 	shmall:       131072	(max amount of shared memory in pages)
> 
> seminfo:
> 	semmni:           50	(# of semaphore identifiers)
> 	semmns:          340	(# of semaphores in system)
> 	semmnu:          150	(# of undo structures in system)
> 	semmsl:          340	(max # of semaphores per id)
> 	semopm:          100	(max # of operations per semop call)
> 	semume:           50	(max # of undo entries per process)
> 	semusz:          632	(size in bytes of undo structure)
> 	semvmx:        32767	(semaphore maximum value)
> 	semaem:        16384	(adjust on exit max value)
> 
> 
> ------------------------------------------------------------------------
> nfsstat
> 
> Client Info:
> Rpc Counts:
>   Getattr   Setattr    Lookup  Readlink      Read     Write    Create   
>  Remove
>         2         0         0         0         0         0         0   
>       0
>    Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus   
>  Access
>         0         0         0         0         0         0         0   
>       0
>     Mknod    Fsstat    Fsinfo  PathConf    Commit
>         0         1         1         0         0
> Rpc Info:
>  TimedOut   Invalid X Replies   Retries  Requests
>         0         0         0         0         4
> Cache Info:
> Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW Hits   
>  Misses
>         0         0         0         0         0         0         0   
>       0
> BioRLHits    Misses BioD Hits    Misses DirE Hits    Misses Accs Hits   
>  Misses
>         0         0         0         0         0         0         0   
>       0
> 
> Server Info:
>   Getattr   Setattr    Lookup  Readlink      Read     Write    Create   
>  Remove
>         0         0         0         0         0         0         0   
>       0
>    Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus   
>  Access
>         0         0         0         0         0         0         0   
>       0
>     Mknod    Fsstat    Fsinfo  PathConf    Commit
>         0         0         0         0         0
> Server Ret-Failed
>                 0
> Server Faults
>             0
> Server Cache Stats:
>    Inprog      Idem  Non-idem    Misses
>         0         0         0         0
> Server Write Gathering:
>  WriteOps  WriteRPC   Opsaved
>         0         0         0
> 
> ------------------------------------------------------------------------
> netstat -s
> 
> tcp:
> 	167 packets sent
> 		138 data packets (12023 bytes)
> 		0 data packets (0 bytes) retransmitted
> 		0 data packets unnecessarily retransmitted
> 		0 resends initiated by MTU discovery
> 		21 ack-only packets (4 delayed)
> 		0 URG only packets
> 		0 window probe packets
> 		0 window update packets
> 		8 control packets
> 	196 packets received
> 		137 acks (for 11879 bytes)
> 		1 duplicate ack
> 		0 acks for unsent data
> 		136 packets (16234 bytes) received in-sequence
> 		0 completely duplicate packets (0 bytes)
> 		0 old duplicate packets
> 		0 packets with some dup. data (0 bytes duped)
> 		0 out-of-order packets (0 bytes)
> 		0 packets (0 bytes) of data after window
> 		0 window probes
> 		0 window update packets
> 		0 packets received after close
> 		0 discarded for bad checksums
> 		0 discarded for bad header offset fields
> 		0 discarded because packet too short
> 		0 discarded due to memory problems
> 	5 connection requests
> 	1 connection accept
> 	0 bad connection attempts
> 	0 listen queue overflows
> 	0 ignored RSTs in the windows
> 	6 connections established (including accepts)
> 	12 connections closed (including 0 drops)
> 		0 connections updated cached RTT on close
> 		0 connections updated cached RTT variance on close
> 		0 connections updated cached ssthresh on close
> 	0 embryonic connections dropped
> 	133 segments updated rtt (of 134 attempts)
> 	0 retransmit timeouts
> 		0 connections dropped by rexmit timeout
> 	0 persist timeouts
> 		0 connections dropped by persist timeout
> 	0 Connections (fin_wait_2) dropped because of timeout
> 	0 keepalive timeouts
> 		0 keepalive probes sent
> 		0 connections dropped by keepalive
> 	3 correct ACK header predictions
> 	57 correct data packet header predictions
> 	1 syncache entry added
> 		0 retransmitted
> 		0 dupsyn
> 		0 dropped
> 		1 completed
> 		0 bucket overflow
> 		0 cache overflow
> 		0 reset
> 		0 stale
> 		0 aborted
> 		0 badack
> 		0 unreach
> 		0 zone failures
> 	1 cookie sent
> 	0 cookies received
> 	0 hostcache entries added
> 		0 bucket overflow
> 	0 SACK recovery episodes
> 	0 segment rexmits in SACK recovery episodes
> 	0 byte rexmits in SACK recovery episodes
> 	0 SACK options (SACK blocks) received
> 	0 SACK options (SACK blocks) sent
> 	0 SACK scoreboard overflow
> 	0 packets with ECN CE bit set
> 	0 packets with ECN ECT(0) bit set
> 	0 packets with ECN ECT(1) bit set
> 	0 successful ECN handshakes
> 	0 times ECN reduced the congestion window
> udp:
> 	118 datagrams received
> 	0 with incomplete header
> 	0 with bad data length field
> 	0 with bad checksum
> 	0 with no checksum
> 	0 dropped due to no socket
> 	36 broadcast/multicast datagrams undelivered
> 	0 dropped due to full socket buffers
> 	0 not for hashed pcb
> 	82 delivered
> 	82 datagrams output
> 	0 times multicast source filter matched
> ip:
> 	267 total packets received
> 	0 bad header checksums
> 	0 with size smaller than minimum
> 	0 with data size < data length
> 	0 with ip length > max ip packet size
> 	0 with header length < data size
> 	0 with data length < header length
> 	0 with bad options
> 	0 with incorrect version number
> 	0 fragments received
> 	0 fragments dropped (dup or out of space)
> 	0 fragments dropped after timeout
> 	0 packets reassembled ok
> 	264 packets for this host
> 	3 packets for unknown/unsupported protocol
> 	0 packets forwarded (0 packets fast forwarded)
> 	0 packets not forwardable
> 	0 packets received for unknown multicast group
> 	0 redirects sent
> 	199 packets sent from this host
> 	0 packets sent with fabricated ip header
> 	0 output packets dropped due to no bufs, etc.
> 	0 output packets discarded due to no route
> 	0 output datagrams fragmented
> 	0 fragments created
> 	0 datagrams that can't be fragmented
> 	0 tunneling packets that can't find gif
> 	0 datagrams with bad address in header
> icmp:
> 	0 calls to icmp_error
> 	0 errors not generated in response to an icmp message
> 	0 messages with bad code fields
> 	0 messages less than the minimum length
> 	0 messages with bad checksum
> 	0 messages with bad length
> 	0 multicast echo requests ignored
> 	0 multicast timestamp requests ignored
> 	0 message responses generated
> 	0 invalid return addresses
> 	0 no return routes
> igmp:
> 	3 messages received
> 	0 messages received with too few bytes
> 	0 messages received with wrong TTL
> 	0 messages received with bad checksum
> 	1 V1/V2 membership query received
> 	0 V3 membership queries received
> 	0 membership queries received with invalid field(s)
> 	1 general query received
> 	0 group queries received
> 	0 group-source queries received
> 	0 group-source queries dropped
> 	1 membership report received
> 	0 membership reports received with invalid field(s)
> 	0 membership reports received for groups to which we belong
> 	0 V3 reports received without Router Alert
> 	0 membership reports sent
> arp:
> 	3 ARP requests sent
> 	3 ARP replies sent
> 	26 ARP requests received
> 	7 ARP replies received
> 	33 ARP packets received
> 	0 total packets dropped due to no ARP entry
> 	0 ARP entrys timed out
> 	0 Duplicate IPs seen
> ip6:
> 	50 total packets received
> 	0 with size smaller than minimum
> 	0 with data size < data length
> 	0 with bad options
> 	0 with incorrect version number
> 	0 fragments received
> 	0 fragments dropped (dup or out of space)
> 	0 fragments dropped after timeout
> 	0 fragments that exceeded limit
> 	0 packets reassembled ok
> 	50 packets for this host
> 	0 packets forwarded
> 	0 packets not forwardable
> 	0 redirects sent
> 	57 packets sent from this host
> 	0 packets sent with fabricated ip header
> 	0 output packets dropped due to no bufs, etc.
> 	4 output packets discarded due to no route
> 	0 output datagrams fragmented
> 	0 fragments created
> 	0 datagrams that can't be fragmented
> 	0 packets that violated scope rules
> 	0 multicast packets which we don't join
> 	Input histogram:
> 		UDP: 50
> 	Mbuf statistics:
> 		26 one mbuf
> 		24 one ext mbuf
> 		0 two or more ext mbuf
> 	0 packets whose headers are not contiguous
> 	0 tunneling packets that can't find gif
> 	0 packets discarded because of too many headers
> 	0 failures of source address selection
> 	Source addresses selection rule applied:
> icmp6:
> 	0 calls to icmp6_error
> 	0 errors not generated in response to an icmp6 message
> 	0 errors not generated because of rate limitation
> 	Output histogram:
> 		neighbor solicitation: 1
> 	0 messages with bad code fields
> 	0 messages < minimum length
> 	0 bad checksums
> 	0 messages with bad length
> 	Histogram of error messages to be generated:
> 		0 no route
> 		0 administratively prohibited
> 		0 beyond scope
> 		0 address unreachable
> 		0 port unreachable
> 		0 packet too big
> 		0 time exceed transit
> 		0 time exceed reassembly
> 		0 erroneous header field
> 		0 unrecognized next header
> 		0 unrecognized option
> 		0 redirect
> 		0 unknown
> 	0 message responses generated
> 	0 messages with too many ND options
> 	0 messages with bad ND options
> 	0 bad neighbor solicitation messages
> 	0 bad neighbor advertisement messages
> 	0 bad router solicitation messages
> 	0 bad router advertisement messages
> 	0 bad redirect messages
> 	0 path MTU changes
> rip6:
> 	0 messages received
> 	0 checksum calculations on inbound
> 	0 messages with bad checksum
> 	0 messages dropped due to no socket
> 	0 multicast messages dropped due to no socket
> 	0 messages dropped due to full socket buffers
> 	0 delivered
> 	0 datagrams output
> 
> ------------------------------------------------------------------------
> netstat -m
> 
> 1025/1485/2510 mbufs in use (current/cache/total)
> 1018/608/1626/4019922 mbuf clusters in use (current/cache/total/max)
> 1023/607 mbuf+clusters out of packet secondary zone in use 
> (current/cache)
> 0/3/3/2009961 4k (page size) jumbo clusters in use 
> (current/cache/total/max)
> 0/0/0/1786632 9k jumbo clusters in use (current/cache/total/max)
> 0/0/0/1339972 16k jumbo clusters in use (current/cache/total/max)
> 2292K/1599K/3891K bytes allocated to network (current/cache/total)
> 0/1490/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
> 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
> 0 requests for sfbufs denied
> 0 requests for sfbufs delayed
> 0 requests for I/O initiated by sendfile
> 
> ------------------------------------------------------------------------
> netstat -id
> 
> Name    Mtu Network       Address              Ipkts Ierrs Idrop
> Opkts Oerrs  Coll Drop
> em0    1500 <Link#1>      00:30:48:f2:29:9c      300     0     0
> 205     0     0    0 em0    1500 192.168.200.0 borg
> 245     -     -      199     -     -    - em0    1500 fe80::230:48f
> fe80::230:48ff:fe        0     -     -        3     -     -    - em1*
>  1500 <Link#2>      00:30:48:f2:29:9d        0     0     0        0
>  0     0    0 lo0   16384 <Link#3>                              50
> 0     0       50     0     0    0 lo0   16384 localhost     ::1
>              50     -     -       50     -     -    - lo0   16384
> fe80::1%lo0   fe80::1                  0     -     -        0     -
>  -    - lo0   16384 your-net      localhost                0     -
> -        0     -     -    -
> 
> ------------------------------------------------------------------------
> netstat -anr
> 
> Routing tables
> 
> Internet:
> Destination        Gateway            Flags    Refs      Use  Netif 
> Expire
> default            192.168.200.11     UGS         0       39    em0
> 127.0.0.1          link#3             UH          0        0    lo0
> 192.168.200.0/24   link#1             U           0      160    em0
> 192.168.200.4      link#1             UHS         0        0    lo0
> 
> Internet6:
> Destination                       Gateway                       Flags
>     Netif Expire
> ::/96                             ::1                           UGRS    
>     lo0
> ::1                               link#3                        UH      
>     lo0
> ::ffff:0.0.0.0/96                 ::1                           UGRS    
>     lo0
> fe80::/10                         ::1                           UGRS    
>     lo0
> fe80::%em0/64                     link#1                        U       
>     em0
> fe80::230:48ff:fef2:299c%em0      link#1                        UHS     
>     lo0
> fe80::%lo0/64                     link#3                        U       
>     lo0
> fe80::1%lo0                       link#3                        UHS     
>     lo0
> ff01::%em0/32                     fe80::230:48ff:fef2:299c%em0  U       
>     em0
> ff01::%lo0/32                     ::1                           U       
>     lo0
> ff02::/16                         ::1                           UGRS    
>     lo0
> ff02::%em0/32                     fe80::230:48ff:fef2:299c%em0  U       
>     em0
> ff02::%lo0/32                     ::1                           U       
>     lo0
> 
> ------------------------------------------------------------------------
> netstat -anA
> 
> Active Internet connections (including servers)
> Tcpcb            Proto Recv-Q Send-Q Local Address      Foreign
> Address    (state)
> fffffe025f039c00 tcp4       0      0 192.168.200.4.3351
> 198.20.8.246.80    ESTABLISHED
> fffffe0206042400 tcp4       0      0 127.0.0.1.31416    *.*
>     LISTEN
> fffffe001eff9400 tcp4       0    148 192.168.200.4.22
> 192.168.200.87.498 ESTABLISHED
> fffffe020602e400 tcp4       0      0 *.9101             *.*
>     LISTEN
> fffffe020615e800 tcp4       0      0 *.22               *.*
>     LISTEN
> fffffe020615ec00 tcp6       0      0 *.22               *.*
>     LISTEN
> fffffe020602e800 tcp4       0      0 *.9102             *.*
>     LISTEN
> fffffe023e226400 tcp4       0      0 *.9103             *.*
>     LISTEN
> fffffe020615f000 tcp4       0      0 *.631              *.*
>     LISTEN
> fffffe020615f400 tcp6       0      0 *.631              *.*
>     LISTEN
> fffffe001eff9800 tcp4       0      0 127.0.0.1.587      *.*
>     LISTEN
> fffffe001eff9c00 tcp4       0      0 127.0.0.1.25       *.*
>     LISTEN
> fffffe001effa000 tcp4       0      0 192.168.200.4.587  *.*
>     LISTEN
> fffffe001effa400 tcp4       0      0 192.168.200.4.25   *.*
>     LISTEN
> fffffe001effa800 tcp4       0      0 127.0.0.1.5432     *.*
>     LISTEN
> fffffe001effac00 tcp6       0      0 ::1.5432           *.*
>     LISTEN
> fffffe02063a6000 tcp4       0      0 *.3551             *.*
>     LISTEN
> fffffe020602ec00 tcp6       0      0 *.2049             *.*
>     LISTEN
> fffffe020602f000 tcp4       0      0 *.2049             *.*
>     LISTEN
> fffffe020615f800 tcp4       0      0 *.977              *.*
>     LISTEN
> fffffe020615fc00 tcp6       0      0 *.977              *.*
>     LISTEN
> fffffe001effb000 tcp4       0      0 *.111              *.*
>     LISTEN
> fffffe020602f400 tcp6       0      0 *.111              *.*
>     LISTEN
> fffffe0206042800 tcp4       0      0 192.168.200.4.755
> 192.168.200.23.204 ESTABLISHED
> fffffe001eb7f7a8 udp4       0      0 *.631              *.*
> fffffe001eba77a8 udp6       0      0 ::1.46935          ::1.46935
> fffffe001eb7f930 udp4       0      0 127.0.0.1.123      *.*
> fffffe001eb7fab8 udp6       0      0 fe80:3::1.123      *.*
> fffffe001eb7fc40 udp6       0      0 ::1.123            *.*
> fffffe001eb7fdc8 udp6       0      0 fe80:1::230:48ff.1 *.*
> fffffe001eb04000 udp4       0      0 192.168.200.4.123  *.*
> fffffe001eb04188 udp6       0      0 *.123              *.*
> fffffe001eb04310 udp4       0      0 *.123              *.*
> fffffe001ee6e188 udp6       0      0 *.2049             *.*
> fffffe001ee6e310 udp4       0      0 *.2049             *.*
> fffffe001eb04498 udp4       0      0 *.977              *.*
> fffffe001eb04620 udp6       0      0 *.977              *.*
> fffffe001ee7c188 udp6       0      0 *.*                *.*
> fffffe001eb047a8 udp4       0      0 *.868              *.*
> fffffe001eb04930 udp4       0      0 *.111              *.*
> fffffe001eb04ab8 udp6       0      0 *.1013             *.*
> fffffe001eb04c40 udp6       0      0 *.111              *.*
> fffffe001eb91620 udp4       0      0 *.514              *.*
> fffffe001eb917a8 udp6       0      0 *.514              *.* Active
> UNIX domain sockets
> Address  Type   Recv-Q Send-Q    Inode     Conn     Refs  Nextref Addr
> fffffe0206108780 stream      0      0        0 fffffe001ee823c0
> 0        0
> fffffe001ee823c0 stream      0      0        0 fffffe0206108780
> 0        0
> fffffe0206108960 stream      0      0 fffffe0206b91588        0
> 0        0 /var/run/cups.sock
> fffffe001ee5b000 stream      0      0 fffffe02068f0760        0
> 0        0 /tmp/.s.PGSQL.5432
> fffffe0206109690 stream      0      0 fffffe020610d760        0
> 0        0 /var/run/rpcbind.sock
> fffffe001ee825a0 stream      0      0 fffffe001e71f938        0
> 0        0 /var/run/devd.pipe
> fffffe001ee822d0 dgram       0      0        0        0        0        
> 0
> fffffe001ee824b0 dgram       0      0        0 fffffe02060dd960
> 0 fffffe001ee89000
> fffffe001ee89000 dgram       0      0        0 fffffe02060dd960
> 0 fffffe0206108870
> fffffe0206108870 dgram       0      0        0 fffffe02060dd960
> 0        0
> fffffe02060dd780 dgram       0      0        0 fffffe02060dd870
> 0 fffffe001ee5b870
> fffffe001ee5b870 dgram       0      0        0 fffffe02060dd870
> 0 fffffe001eeb02d0
> fffffe001eeb02d0 dgram       0      0        0 fffffe02060dd870
> 0 fffffe001ee61b40
> fffffe001ee61b40 dgram       0      0        0 fffffe02060dd870
> 0        0
> fffffe02060dd870 dgram       0      0 fffffe001e786938        0
> fffffe02060dd780        0 /var/run/logpriv
> fffffe02060dd960 dgram       0      0 fffffe001e786b10        0
> fffffe001ee824b0        0 /var/run/log
> 
> ------------------------------------------------------------------------
> netstat -aL
> 
> Current listen queue sizes (qlen/incqlen/maxqlen)
> Proto Listen         Local Address tcp4  0/0/128
> localhost.31416 tcp4  0/0/50         *.bacula-dir tcp4  0/0/128
> *.ssh tcp6  0/0/128        *.ssh tcp4  0/0/50         *.bacula-fd tcp4
>  0/0/50         *.bacula-sd tcp4  0/0/128        *.ipp tcp6  0/0/128
>      *.ipp tcp4  0/0/20         localhost.submission tcp4  0/0/20
>    localhost.smtp tcp4  0/0/20         borg.submission tcp4  0/0/20
>      borg.smtp tcp4  0/0/128        localhost.postgresql tcp6  0/0/128
>        localhost.postgresql tcp4  0/0/5          *.3551 tcp6  0/0/5
>       *.nfsd tcp4  0/0/5          *.nfsd tcp4  0/0/128        *.977
> tcp6  0/0/128        *.977 tcp4  0/0/128        *.sunrpc tcp6  0/0/128
>        *.sunrpc unix  0/0/128        /var/run/cups.sock
> unix  0/0/128        /tmp/.s.PGSQL.5432
> unix  0/0/128        /var/run/rpcbind.sock
> unix  0/0/4          /var/run/devd.pipe
> 
> ------------------------------------------------------------------------
> fstat
> 
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read file 10 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read file 10 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read file 10 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read file 10 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read file 10 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read file 10 at 0xffff
> fstat: can't read file 13 at 0xfffffffffffffff
> fstat: can't read file 14 at 0x78
> fstat: can't read file 16 at 0xffff
> fstat: can't read file 19 at 0xfffffffffffffff
> fstat: can't read file 20 at 0x78
> fstat: can't read file 22 at 0xffff
> fstat: can't read file 23 at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read file 10 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read file 7 at 0xfffffffffffffff
> fstat: can't read file 8 at 0x78
> fstat: can't read file 10 at 0xffff
> fstat: can't read file 13 at 0xfffffffffffffff
> fstat: can't read file 14 at 0x78
> fstat: can't read file 16 at 0xffff
> fstat: can't read file 19 at 0xfffffffffffffff
> fstat: can't read file 20 at 0x78
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read file 1 at 0xfffffffffffffff
> fstat: can't read file 2 at 0x78
> fstat: can't read file 4 at 0xffff
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> fstat: can't read znode_phys at 0x1
> USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
> root     cpucontrol  1378 root -         -       error    -
> root     cpucontrol  1378   wd -         -       error    -
> root     cpucontrol  1378 text -         -       error    -
> root     cpucontrol  1378 ctty /dev        169 crw--w----   pts/0 rw
> root     cpucontrol  1378    0 /dev        169 crw--w----   pts/0 rw
> root     sh          1372 root -         -       error    -
> root     sh          1372   wd -         -       error    -
> root     sh          1372 text -         -       error    -
> root     sh          1372 ctty /dev        169 crw--w----   pts/0 rw
> root     sh          1372    0 /dev        169 crw--w----   pts/0 rw
> root     sh          1372    6 /dev        169 crw--w----   pts/0 rw
> boinc    wcgrid_faah_7.15_i  1362 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1362   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1362 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1362    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1362    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1361 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1361   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1361 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1361    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1361    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1360 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1360   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1360 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1360    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1360    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1359 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1359   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1359 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1359    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1359    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1358 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1358   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1358 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1358    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1358    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1357 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1357   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1357 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1357    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1357    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1356 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1356   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1356 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1356    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1356    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_cep2_6.40_i  1355 root -         -       error    -
> boinc    wcgrid_cep2_6.40_i  1355   wd -         -       error    -
> boinc    wcgrid_cep2_6.40_i  1355 text -         -       error    -
> boinc    wcgrid_cep2_6.40_i  1355    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_cep2_6.40_i  1355    6 -         -       error    -
> boinc    wcgrid_faah_7.15_i  1354 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1354   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1354 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1354    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1354    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_cep2_6.40_i  1353 root -         -       error    -
> boinc    wcgrid_cep2_6.40_i  1353   wd -         -       error    -
> boinc    wcgrid_cep2_6.40_i  1353 text -         -       error    -
> boinc    wcgrid_cep2_6.40_i  1353    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_cep2_6.40_i  1353    6 -         -       error    -
> boinc    wcgrid_faah_7.15_i  1352 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1352   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1352 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1352    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1352    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1351 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1351   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1351 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1351    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1351    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1350 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1350   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1350 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1350    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1350    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1349 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1349   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1349 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1349    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1349    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1348 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1348   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1348 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1348    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1348    6 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_cep2_6.40_i  1347 root -         -       error    -
> boinc    wcgrid_cep2_6.40_i  1347   wd -         -       error    -
> boinc    wcgrid_cep2_6.40_i  1347 text -         -       error    -
> boinc    wcgrid_cep2_6.40_i  1347    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_cep2_6.40_i  1347    6 -         -       error    -
> boinc    wcgrid_faah_7.15_i  1346 root -         -       error    -
> boinc    wcgrid_faah_7.15_i  1346   wd -         -       error    -
> boinc    wcgrid_faah_7.15_i  1346 text -         -       error    -
> boinc    wcgrid_faah_7.15_i  1346    0 /dev         30 crw-rw-rw-    
> null rw
> boinc    wcgrid_faah_7.15_i  1346    6 /dev         30 crw-rw-rw-    
> null rw
> root     sh          1345 root -         -       error    -
> root     sh          1345   wd -         -       error    -
> root     sh          1345 text -         -       error    -
> root     sh          1345 ctty /dev        169 crw--w----   pts/0 rw
> root     sh          1345    0 /dev        169 crw--w----   pts/0 rw
> root     sh          1345    6 /dev        169 crw--w----   pts/0 rw
> root     sudo        1344 root -         -       error    -
> root     sudo        1344   wd -         -       error    -
> root     sudo        1344 text -         -       error    -
> root     sudo        1344 ctty /dev        169 crw--w----   pts/0 rw
> root     sudo        1344    0 /dev        169 crw--w----   pts/0 rw
> ler      sh          1342 root -         -       error    -
> ler      sh          1342   wd -         -       error    -
> ler      sh          1342 text -         -       error    -
> ler      sh          1342 ctty /dev        169 crw--w----   pts/0 rw
> ler      sh          1342    0 /dev        169 crw--w----   pts/0 rw
> ler      sh          1342    6 /dev        169 crw--w----   pts/0 rw
> ler      sshd        1341 root -         -       error    -
> ler      sshd        1341   wd -         -       error    -
> ler      sshd        1341 text -         -       error    -
> ler      sshd        1341    0 /dev         30 crw-rw-rw-    null rw
> ler      sshd        1341    6 /dev         30 crw-rw-rw-    null rw
> root     sshd        1339 root -         -       error    -
> root     sshd        1339   wd -         -       error    -
> root     sshd        1339 text -         -       error    -
> root     sshd        1339    0 /dev         30 crw-rw-rw-    null  r
> root     sshd        1339    6 /dev         30 crw-rw-rw-    null rw
> root     getty       1337 root -         -       error    -
> root     getty       1337   wd -         -       error    -
> root     getty       1337 text -         -       error    -
> root     getty       1337 ctty /dev         83 crw-------   ttyv7 rw
> root     getty       1337    0 /dev         83 crw-------   ttyv7 rw
> root     getty       1336 root -         -       error    -
> root     getty       1336   wd -         -       error    -
> root     getty       1336 text -         -       error    -
> root     getty       1336 ctty /dev         82 crw-------   ttyv6 rw
> root     getty       1336    0 /dev         82 crw-------   ttyv6 rw
> root     getty       1335 root -         -       error    -
> root     getty       1335   wd -         -       error    -
> root     getty       1335 text -         -       error    -
> root     getty       1335 ctty /dev         81 crw-------   ttyv5 rw
> root     getty       1335    0 /dev         81 crw-------   ttyv5 rw
> root     getty       1334 root -         -       error    -
> root     getty       1334   wd -         -       error    -
> root     getty       1334 text -         -       error    -
> root     getty       1334 ctty /dev         80 crw-------   ttyv4 rw
> root     getty       1334    0 /dev         80 crw-------   ttyv4 rw
> root     getty       1333 root -         -       error    -
> root     getty       1333   wd -         -       error    -
> root     getty       1333 text -         -       error    -
> root     getty       1333 ctty /dev         79 crw-------   ttyv3 rw
> root     getty       1333    0 /dev         79 crw-------   ttyv3 rw
> root     getty       1332 root -         -       error    -
> root     getty       1332   wd -         -       error    -
> root     getty       1332 text -         -       error    -
> root     getty       1332 ctty /dev         78 crw-------   ttyv2 rw
> root     getty       1332    0 /dev         78 crw-------   ttyv2 rw
> root     getty       1331 root -         -       error    -
> root     getty       1331   wd -         -       error    -
> root     getty       1331 text -         -       error    -
> root     getty       1331 ctty /dev         77 crw-------   ttyv1 rw
> root     getty       1331    0 /dev         77 crw-------   ttyv1 rw
> root     getty       1330 root -         -       error    -
> root     getty       1330   wd -         -       error    -
> root     getty       1330 text -         -       error    -
> root     getty       1330 ctty /dev         76 crw-------   ttyv0 rw
> root     getty       1330    0 /dev         76 crw-------   ttyv0 rw
> root     sleep       1328 root -         -       error    -
> root     sleep       1328   wd -         -       error    -
> root     sleep       1328 text -         -       error    -
> root     sleep       1328    0 /dev         30 crw-rw-rw-    null  r
> root     logger      1327 root -         -       error    -
> root     logger      1327   wd -         -       error    -
> root     logger      1327 text -         -       error    -
> root     logger      1327    0* pipe fffffe001e7bfba0 <->
> fffffe001e7bfd00      0 rw
> root     sh          1326 root -         -       error    -
> root     sh          1326   wd -         -       error    -
> root     sh          1326 text -         -       error    -
> root     sh          1326    0 /dev         30 crw-rw-rw-    null  r
> root     inetd       1310 root -         -       error    -
> root     inetd       1310   wd -         -       error    -
> root     inetd       1310 text -         -       error    -
> root     inetd       1310    0 /dev         30 crw-rw-rw-    null rw
> root     cron        1288 root -         -       error    -
> root     cron        1288   wd -         -       error    -
> root     cron        1288 text -         -       error    -
> root     cron        1288    0 /dev         30 crw-rw-rw-    null rw
> root     sshd        1276 root -         -       error    -
> root     sshd        1276   wd -         -       error    -
> root     sshd        1276 text -         -       error    -
> root     sshd        1276    0 /dev         30 crw-rw-rw-    null rw
> bacula   bacula-dir  1271 root -         -       error    -
> bacula   bacula-dir  1271   wd -         -       error    -
> bacula   bacula-dir  1271 text -         -       error    -
> bacula   bacula-dir  1271    0 /dev         30 crw-rw-rw-    null  r
> root     bacula-fd   1268 root -         -       error    -
> root     bacula-fd   1268   wd -         -       error    -
> root     bacula-fd   1268 text -         -       error    -
> root     bacula-fd   1268    0 /dev         30 crw-rw-rw-    null  r
> bacula   bacula-sd   1265 root -         -       error    -
> bacula   bacula-sd   1265   wd -         -       error    -
> bacula   bacula-sd   1265 text -         -       error    -
> bacula   bacula-sd   1265    0 /dev         30 crw-rw-rw-    null  r
> boinc    boinc_client  1261 root -         -       error    -
> boinc    boinc_client  1261   wd -         -       error    -
> boinc    boinc_client  1261 text -         -       error    -
> boinc    boinc_client  1261    0 /dev         30 crw-rw-rw-    null rw
> boinc    boinc_client  1261    6 -         -       error    -
> root     cupsd       1255 root -         -       error    -
> root     cupsd       1255   wd -         -       error    -
> root     cupsd       1255 text -         -       error    -
> root     cupsd       1255    0 /dev         30 crw-rw-rw-    null  r
> root     cupsd       1255    6 /dev         30 crw-rw-rw-    null  w
> root     cupsd       1255   12 /dev         30 crw-rw-rw-    null  w
> mailnull exim-4.80.1-2  1246 root -         -       error    -
> mailnull exim-4.80.1-2  1246   wd -         -       error    -
> mailnull exim-4.80.1-2  1246 text -         -       error    -
> mailnull exim-4.80.1-2  1246    0 /dev         30 crw-rw-rw-    null rw
> mailnull exim-4.80.1-2  1246    6 /dev         30 crw-rw-rw-    null rw
> pgsql    postgres    1156 root -         -       error    -
> pgsql    postgres    1156   wd -         -       error    -
> pgsql    postgres    1156 text -         -       error    -
> pgsql    postgres    1156    0 /dev         30 crw-rw-rw-    null  r
> pgsql    postgres    1156    6 -         -       error    -
> pgsql    postgres    1155 root -         -       error    -
> pgsql    postgres    1155   wd -         -       error    -
> pgsql    postgres    1155 text -         -       error    -
> pgsql    postgres    1155    0 /dev         30 crw-rw-rw-    null  r
> pgsql    postgres    1155    6 -         -       error    -
> pgsql    postgres    1154 root -         -       error    -
> pgsql    postgres    1154   wd -         -       error    -
> pgsql    postgres    1154 text -         -       error    -
> pgsql    postgres    1154    0 /dev         30 crw-rw-rw-    null  r
> pgsql    postgres    1154    6 -         -       error    -
> pgsql    postgres    1153 root -         -       error    -
> pgsql    postgres    1153   wd -         -       error    -
> pgsql    postgres    1153 text -         -       error    -
> pgsql    postgres    1153    0 /dev         30 crw-rw-rw-    null  r
> pgsql    postgres    1153    6 -         -       error    -
> pgsql    postgres    1149 root -         -       error    -
> pgsql    postgres    1149   wd -         -       error    -
> pgsql    postgres    1149 text -         -       error    -
> pgsql    postgres    1149    0 /dev         30 crw-rw-rw-    null  r
> pgsql    postgres    1149    6 -         -       error    -
> root     smartd      1140 root -         -       error    -
> root     smartd      1140   wd -         -       error    -
> root     smartd      1140 text -         -       error    -
> root     smartd      1140    0 /dev         30 crw-rw-rw-    null rw
> root     perl        1136 root -         -       error    -
> root     perl        1136   wd -         -       error    -
> root     perl        1136 text -         -       error    -
> root     perl        1136    0 -         -       error    -
> root     ng_queue    1124 root -         -       error    -
> root     ng_queue    1124   wd -         -       error    -
> root     ntpd        1103 root -         -       error    -
> root     ntpd        1103   wd -         -       error    -
> root     ntpd        1103 text -         -       error    -
> root     ntpd        1103    0 /dev         30 crw-rw-rw-    null rw
> root     ntpd        1103    6 /dev         30 crw-rw-rw-    null rw
> root     ntpd        1103   12 /dev         30 crw-rw-rw-    null rw
> root     ntpd        1103   18* local dgram fffffe001eeb02d0 <->
> fffffe02060dd870
> root     apcupsd     1072 root -         -       error    -
> root     apcupsd     1072   wd -         -       error    -
> root     apcupsd     1072 text -         -       error    -
> root     apcupsd     1072    0 /dev         30 crw-rw-rw-    null  r
> root     apcupsd     1072    6 /dev         30 crw-rw-rw-    null  r
> root     nfsd        1064 root -         -       error    -
> root     nfsd        1064   wd -         -       error    -
> root     nfsd        1064 text -         -       error    -
> root     nfsd        1064    0 /dev         30 crw-rw-rw-    null rw
> root     nfsd        1062 root -         -       error    -
> root     nfsd        1062   wd -         -       error    -
> root     nfsd        1062 text -         -       error    -
> root     nfsd        1062    0 /dev         30 crw-rw-rw-    null rw
> root     nfsd        1062    6 /dev         30 crw-rw-rw-    null rw
> root     mountd      1056 root -         -       error    -
> root     mountd      1056   wd -         -       error    -
> root     mountd      1056 text -         -       error    -
> root     mountd      1056    0 /dev         30 crw-rw-rw-    null rw
> root     mountd      1056    6 /dev         30 crw-rw-rw-    null rw
> root     rpcbind     1021 root -         -       error    -
> root     rpcbind     1021   wd -         -       error    -
> root     rpcbind     1021 text -         -       error    -
> root     rpcbind     1021    0 /dev         30 crw-rw-rw-    null rw
> root     rpcbind     1021    6 /dev         30 crw-rw-rw-    null rw
> root     watchdogd   1007 root -         -       error    -
> root     watchdogd   1007   wd -         -       error    -
> root     watchdogd   1007 text -         -       error    -
> root     watchdogd   1007    0 /dev         30 crw-rw-rw-    null rw
> root     syslogd     1004 root -         -       error    -
> root     syslogd     1004   wd -         -       error    -
> root     syslogd     1004 text -         -       error    -
> root     syslogd     1004    0 /dev         30 crw-rw-rw-    null rw
> root     syslogd     1004    6 /dev         30 crw-rw-rw-    null rw
> root     syslogd     1004   12 /dev         30 crw-rw-rw-    null rw
> root     syslogd     1004   18 -         -       error    -
> root     devd         866 root -         -       error    -
> root     devd         866   wd -         -       error    -
> root     devd         866 text -         -       error    -
> root     devd         866    0 /dev         30 crw-rw-rw-    null rw
> root     devd         866    6 /dev         30 crw-rw-rw-    null rw
> root     moused       847 root -         -       error    -
> root     moused       847   wd -         -       error    -
> root     moused       847 text -         -       error    -
> root     moused       847    0 /dev         30 crw-rw-rw-    null rw
> root     init           1 root -         -       error    -
> root     init           1   wd -         -       error    -
> root     init           1 text -         -       error    -
> root     kernel         0 root -         -       error    -
> root     kernel         0   wd -         -       error    -
> 
> ------------------------------------------------------------------------
> dmesg
> 
> Copyright (c) 1992-2013 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 
> 1994
> 	The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 10.0-CURRENT #16: Sat Aug 10 13:45:06 CDT 2013
>     root_at_borg.lerctr.org:/usr/obj/usr/src/sys/BORG-DTRACE amd64
> FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610
> CPU: Intel(R) Xeon(R) CPU           E5410  _at_ 2.33GHz (2327.55-MHz 
> K8-class CPU)
>   Origin = "GenuineIntel"  Id = 0x10676  Family = 0x6  Model = 0x17
> Stepping = 6
> 
> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
> 
> Features2=0xce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1>
>   AMD Features=0x20100800<SYSCALL,NX,LM>
>   AMD Features2=0x1<LAHF>
>   TSC: P-state invariant, performance statistics
> real memory  = 68719476736 (65536 MB)
> avail memory = 65660014592 (62618 MB)
> Event timer "LAPIC" quality 400
> ACPI APIC Table: <PTLTD  	 APIC  >
> FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
> FreeBSD/SMP: 2 package(s) x 4 core(s)
>  cpu0 (BSP): APIC ID:  0
>  cpu1 (AP): APIC ID:  1
>  cpu2 (AP): APIC ID:  2
>  cpu3 (AP): APIC ID:  3
>  cpu4 (AP): APIC ID:  4
>  cpu5 (AP): APIC ID:  5
>  cpu6 (AP): APIC ID:  6
>  cpu7 (AP): APIC ID:  7
> ioapic0 <Version 2.0> irqs 0-23 on motherboard
> ioapic1 <Version 2.0> irqs 24-47 on motherboard
> kbd1 at kbdmux0
> netmap: loaded module
> random: <Software, Yarrow> initialized
> cryptosoft0: <software crypto> on motherboard
> acpi0: <PTLTD   RSDT> on motherboard
> acpi0: Power Button (fixed)
> unknown: I/O range not supported
> cpu0: <ACPI CPU> on acpi0
> cpu1: <ACPI CPU> on acpi0
> cpu2: <ACPI CPU> on acpi0
> cpu3: <ACPI CPU> on acpi0
> cpu4: <ACPI CPU> on acpi0
> cpu5: <ACPI CPU> on acpi0
> cpu6: <ACPI CPU> on acpi0
> cpu7: <ACPI CPU> on acpi0
> hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff irq 0,8 
> on acpi0
> Timecounter "HPET" frequency 14318180 Hz quality 950
> Event timer "HPET" frequency 14318180 Hz quality 350
> Event timer "HPET1" frequency 14318180 Hz quality 340
> Event timer "HPET2" frequency 14318180 Hz quality 340
> atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
> Event timer "RTC" frequency 32768 Hz quality 0
> attimer0: <AT timer> port 0x40-0x43,0x50-0x53 on acpi0
> Timecounter "i8254" frequency 1193182 Hz quality 0
> Event timer "i8254" frequency 1193182 Hz quality 100
> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
> pci0: <ACPI PCI bus> on pcib0
> pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
> pci1: <ACPI PCI bus> on pcib1
> pcib2: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci1
> pci2: <ACPI PCI bus> on pcib2
> pcib3: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci2
> pci3: <ACPI PCI bus> on pcib3
> pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3
> pci4: <ACPI PCI bus> on pcib4
> pcib5: <ACPI PCI-PCI bridge> at device 0.2 on pci3
> pci5: <ACPI PCI bus> on pcib5
> pcib6: <ACPI PCI-PCI bridge> irq 18 at device 2.0 on pci2
> pci6: <ACPI PCI bus> on pcib6
> em0: <Intel(R) PRO/1000 Network Connection 7.3.8> port 0x2000-0x201f
> mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0.0
> on pci6
> em0: Using an MSI interrupt
> em0: Ethernet address: 00:30:48:f2:29:9c
> 001.000009 netmap_attach [2244] success for em0
> em1: <Intel(R) PRO/1000 Network Connection 7.3.8> port 0x2020-0x203f
> mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0.1
> on pci6
> em1: Using an MSI interrupt
> em1: Ethernet address: 00:30:48:f2:29:9d
> 001.000010 netmap_attach [2244] success for em1
> pcib7: <ACPI PCI-PCI bridge> at device 0.3 on pci1
> pci7: <ACPI PCI bus> on pcib7
> pcib8: <ACPI PCI-PCI bridge> at device 4.0 on pci0
> pci8: <ACPI PCI bus> on pcib8
> vgapci0: <VGA-compatible display> port 0x3000-0x307f mem
> 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff irq
> 16 at device 0.0 on pci8
> nvidia0: <GeForce GT 610> on vgapci0
> vgapci0: child nvidia0 requested pci_enable_io
> vgapci0: child nvidia0 requested pci_enable_io
> pci8: <multimedia, HDA> at device 0.1 (no driver attached)
> pcib9: <ACPI PCI-PCI bridge> at device 6.0 on pci0
> pci9: <ACPI PCI bus> on pcib9
> pci0: <base peripheral> at device 8.0 (no driver attached)
> pcib10: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
> pci10: <ACPI PCI bus> on pcib10
> pcib11: <PCI-PCI bridge> irq 16 at device 0.0 on pci10
> pci11: <PCI bus> on pcib11
> pcm0: <Envy24HT audio (Generic)> port 0x4080-0x409f,0x4000-0x407f irq
> 16 at device 0.0 on pci11
> pcm0: system configuration
>   SubVendorID: 0x1412, SubDeviceID: 0x2403
>   XIN2 Clock Source: 24.576MHz(96kHz*256)
>   MPU-401 UART(s) #: not implemented
>   ADC #: 1 and SPDIF receiver connected
>   DAC #: 4
>   Multi-track converter type: AC'97(SDATA_OUT:packed)
>   S/PDIF(IN/OUT): 1/1 ID# 0x00
>   GPIO(mask/dir/state): 0xff/0xff/0xff
> uhci0: <Intel 631XESB/632XESB/3100 USB controller USB-1> port
> 0x1800-0x181f irq 17 at device 29.0 on pci0
> usbus0 on uhci0
> uhci1: <Intel 631XESB/632XESB/3100 USB controller USB-2> port
> 0x1820-0x183f irq 19 at device 29.1 on pci0
> usbus1 on uhci1
> uhci2: <Intel 631XESB/632XESB/3100 USB controller USB-3> port
> 0x1840-0x185f irq 18 at device 29.2 on pci0
> usbus2 on uhci2
> ehci0: <Intel 63XXESB USB 2.0 controller> mem 0xd9600400-0xd96007ff
> irq 17 at device 29.7 on pci0
> usbus3: EHCI version 1.0
> usbus3 on ehci0
> pcib12: <ACPI PCI-PCI bridge> at device 30.0 on pci0
> pci12: <ACPI PCI bus> on pcib12
> vgapci1: <VGA-compatible display> port 0x5000-0x50ff mem
> 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 on
> pci12
> isab0: <PCI-ISA bridge> at device 31.0 on pci0
> isa0: <ISA bus> on isab0
> atapci0: <Intel 63XXESB2 UDMA100 controller> port
> 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on
> pci0
> ata0: <ATA channel> at channel 0 on atapci0
> ahci0: <Intel ESB2 AHCI SATA controller> port
> 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f
> mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0
> ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported
> ahcich0: <AHCI channel> at channel 0 on ahci0
> ahcich1: <AHCI channel> at channel 1 on ahci0
> ahcich2: <AHCI channel> at channel 2 on ahci0
> ahcich3: <AHCI channel> at channel 3 on ahci0
> ahcich4: <AHCI channel> at channel 4 on ahci0
> ahcich5: <AHCI channel> at channel 5 on ahci0
> ichsmb0: <Intel 631xESB/6321ESB (ESB2) SMBus controller> port
> 0x1100-0x111f irq 19 at device 31.3 on pci0
> smbus0: <System Management Bus> on ichsmb0
> acpi_button0: <Power Button> on acpi0
> atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> atkbd0: [GIANT-LOCKED]
> uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
> uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
> fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on 
> acpi0
> fd0: <1440-KB 3.5" drive> on fdc0 drive 0
> ppc0: <Parallel port> port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0
> ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
> ppc0: FIFO with 16/16/9 bytes threshold
> ppbus0: <Parallel port bus> on ppc0
> lpt0: <Printer> on ppbus0
> lpt0: Interrupt-driven port
> ppi0: <Parallel I/O> on ppbus0
> ichwd0 on isa0
> sc0: <System console> at flags 0x100 on isa0
> sc0: VGA <16 virtual consoles, flags=0x300>
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on 
> isa0
> coretemp0: <CPU On-Die Thermal Sensors> on cpu0
> est0: <Enhanced SpeedStep Frequency Control> on cpu0
> p4tcc0: <CPU Frequency Thermal Control> on cpu0
> coretemp1: <CPU On-Die Thermal Sensors> on cpu1
> est1: <Enhanced SpeedStep Frequency Control> on cpu1
> p4tcc1: <CPU Frequency Thermal Control> on cpu1
> coretemp2: <CPU On-Die Thermal Sensors> on cpu2
> est2: <Enhanced SpeedStep Frequency Control> on cpu2
> p4tcc2: <CPU Frequency Thermal Control> on cpu2
> coretemp3: <CPU On-Die Thermal Sensors> on cpu3
> est3: <Enhanced SpeedStep Frequency Control> on cpu3
> p4tcc3: <CPU Frequency Thermal Control> on cpu3
> coretemp4: <CPU On-Die Thermal Sensors> on cpu4
> est4: <Enhanced SpeedStep Frequency Control> on cpu4
> p4tcc4: <CPU Frequency Thermal Control> on cpu4
> coretemp5: <CPU On-Die Thermal Sensors> on cpu5
> est5: <Enhanced SpeedStep Frequency Control> on cpu5
> p4tcc5: <CPU Frequency Thermal Control> on cpu5
> coretemp6: <CPU On-Die Thermal Sensors> on cpu6
> est6: <Enhanced SpeedStep Frequency Control> on cpu6
> p4tcc6: <CPU Frequency Thermal Control> on cpu6
> coretemp7: <CPU On-Die Thermal Sensors> on cpu7
> est7: <Enhanced SpeedStep Frequency Control> on cpu7
> p4tcc7: <CPU Frequency Thermal Control> on cpu7
> ZFS filesystem version: 5
> ZFS storage pool version: features support (5000)
> Timecounters tick every 1.000 msec
> vboxdrv: fAsync=0 offMin=0x395 offMax=0x50f
> usbus0: 12Mbps Full Speed USB v1.0
> usbus1: 12Mbps Full Speed USB v1.0
> usbus2: 12Mbps Full Speed USB v1.0
> usbus3: 480Mbps High Speed USB v2.0
> ugen0.1: <Intel> at usbus0
> uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on 
> usbus0
> ugen1.1: <Intel> at usbus1
> uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on 
> usbus1
> ugen2.1: <Intel> at usbus2
> uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on 
> usbus2
> ugen3.1: <Intel> at usbus3
> uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on 
> usbus3
> ata0: DMA limited to UDMA33, controller found non-ATA66 cable
> ada0 at ahcich0 bus 0 scbus1 target 0 lun 0
> ada0: <ST2000DL003-9VT166 CC3D> ATA-8 SATA 3.x device
> ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
> ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
> ada0: quirks=0x1<4K>
> ada0: Previously was known as ad4
> ada1 at ahcich1 bus 0 scbus2 target 0 lun 0
> ada1: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
> ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
> ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
> ada1: quirks=0x1<4K>
> ada1: Previously was known as ad6
> ada2 at ahcich2 bus 0 scbus3 target 0 lun 0
> ada2: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
> ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
> ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
> ada2: quirks=0x1<4K>
> ada2: Previously was known as ad8
> ada3 at ahcich3 bus 0 scbus4 target 0 lun 0
> ada3: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
> ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
> ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
> ada3: quirks=0x1<4K>
> ada3: Previously was known as ad10
> ada4 at ahcich4 bus 0 scbus5 target 0 lun 0
> ada4: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
> ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
> ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
> ada4: quirks=0x1<4K>
> ada4: Previously was known as ad12
> ada5 at ahcich5 bus 0 scbus6 target 0 lun 0
> ada5: <ST2000DL003-9VT166 CC3D> ATA-8 SATA 3.x device
> ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
> ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
> ada5: quirks=0x1<4K>
> ada5: Previously was known as ad14
> cd0 at ata0 bus 0 scbus0 target 0 lun 0
> cd0: <Memorex DVD+-RAM 510L v1 MWS7> Removable CD-ROM SCSI-0 device
> cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
> cd0: Attempt to query device size failed: NOT READY, Medium not present
> SMP: AP CPU #6 Launched!
> SMP: AP CPU #1 Launched!
> SMP: AP CPU #3 Launched!
> SMP: AP CPU #2 Launched!
> SMP: AP CPU #7 Launched!
> SMP: AP CPU #4 Launched!
> SMP: AP CPU #5 Launched!
> uhub1: 2 ports with 2 removable, self powered
> uhub0: 2 ports with 2 removable, self powered
> uhub2: 2 ports with 2 removable, self powered
> Root mount waiting for: usbus3
> uhub3: 6 ports with 6 removable, self powered
> Root mount waiting for: usbus3
> Root mount waiting for: usbus3
> ugen3.2: <Peppercon AG> at usbus3
> ukbd0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on 
> usbus3
> kbd2 at ukbd0
> Trying to mount root from zfs:zroot/ROOT/default []...
> ugen0.2: <Microsoft> at usbus0
> ugen0.3: <American Power Conversion> at usbus0
> ffclock reset: HPET (14318180 Hz), time = 1376160821.500000000
> Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3.
> Setting hostid: 0xf53a926e.
> Entropy harvesting: interrupts ethernet point_to_point kickstart.
> Starting file system checks:
> Mounting local file systems:.
> Writing entropy file:.
> Setting hostname: borg.lerctr.org.
> Starting Network: lo0 em0 em1.
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
> 	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
> 	inet6 ::1 prefixlen 128
> 	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
> 	inet 127.0.0.1 netmask 0xff000000
> 	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 
> 1500
> 	options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
> 	ether 00:30:48:f2:29:9c
> 	inet 192.168.200.4 netmask 0xffffff00 broadcast 192.168.200.255
> 	inet6 fe80::230:48ff:fef2:299c%em0 prefixlen 64 scopeid 0x1
> 	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> 	media: Ethernet autoselect
> 	status: no carrier
> em1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
> 	options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
> 	ether 00:30:48:f2:29:9d
> 	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> 	media: Ethernet autoselect
> 	status: no carrier
> Starting devd.
> Starting Network: em1.
> em1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
> 	options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
> 	ether 00:30:48:f2:29:9d
> 	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> 	media: Ethernet autoselect
> 	status: no carrier
> uhid0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on 
> usbus3
> ums0: <Microsoft Microsoft Trackball Optical, class 0/0, rev
> 1.10/1.21, addr 2> on usbus0
> ums0: 5 buttons and [XYZ] coordinates ID=0
> Starting ums0 moused.
> add net default: gateway 192.168.200.11
> add net fe80::: gateway ::1
> add net ff02::: gateway ::1
> add net ::ffff:0.0.0.0: gateway ::1
> add net ::0.0.0.0: gateway ::1
> Mounting NFS file systems:.
> ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib
> /usr/local/kde4/lib /usr/local/lib/compat /usr/local/lib/dbmail
> /usr/local/lib/event2 /usr/local/lib/pth /usr/local/lib/qt4
> /usr/local/lib/virtualbox
> 32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat
> Creating and/or trimming log files.
> Starting syslogd.
> Starting watchdogd.
> No core dumps found.
> Additional ABI support: linux.
> Starting rpcbind.
> NFS access cache time=60
> Clearing /tmp (X related).
> Starting mountd.
> NFSv4 is disabled
> Starting nfsd.
> Starting apcupsd.
> Updating motd:.
> Mounting late file systems:.
> Starting ntpd.
> Starting sshblock.
> Starting smartd.
> Starting exim.
> Starting cupsd.
> Starting boinc_client.
> Starting bacula_sd.
> Starting bacula_fd.
> Starting bacula_dir.
> Performing sanity check on sshd configuration.
> Starting sshd.
> Configuring syscons: blanktime.
> Starting cron.
> mixer: WRITE_MIXER: Device not configured
> mixer: WRITE_MIXER: Device not configured
> mixer: WRITE_MIXER: Device not configured
> mixer: WRITE_MIXER: Device not configured
> mixer: WRITE_MIXER: Device not configured
> mixer: WRITE_MIXER: Device not configured
> Starting inetd.
> Starting background file system checks in 60 seconds.
> 
> Sat Aug 10 13:54:18 CDT 2013
> 
> 
> Fatal trap 12: page fault while in kernel mode
> cpuid = 0; apic id = 00
> fault virtual address	= 0x5e
> fault code		= supervisor read data, page not present
> instruction pointer	= 0x20:0xffffffff80709be9
> stack pointer	        = 0x28:0xffffff900d55c860
> frame pointer	        = 0x28:0xffffff900d55c870
> code segment		= base 0x0, limit 0xfffff, type 0x1b
> 			= DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags	= interrupt enabled, resume, IOPL = 0
> current process		= 1378 (cpucontrol)
> trap number		= 12
> panic: page fault
> cpuid = 0
> Uptime: 1m48s
> Dumping 2337 out of 64479 
> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
> 
> ------------------------------------------------------------------------
> kernel config
> 
> options	CONFIG_AUTOGENERATED
> ident	BORG-DTRACE
> machine	amd64
> cpu	HAMMER
> makeoptions	WITH_CTF=1
> makeoptions	DEBUG=-g
> options	FFCLOCK
> options	USB_DEBUG
> options	RDRAND_RNG
> options	PADLOCK_RNG
> options	ATH_ENABLE_11N
> options	AH_AR5416_INTERRUPT_MITIGATION
> options	AH_SUPPORT_AR5416
> options	IEEE80211_SUPPORT_MESH
> options	IEEE80211_AMPDU_AGE
> options	IEEE80211_DEBUG
> options	SC_PIXEL_MODE
> options	VESA
> options	AHD_REG_PRETTY_PRINT
> options	AHC_REG_PRETTY_PRINT
> options	ATA_STATIC_ID
> options	SMP
> options	MALLOC_DEBUG_MAXZONES=8
> options	KDB_TRACE
> options	INCLUDE_CONFIG_FILE
> options	DDB_CTF
> options	KDTRACE_HOOKS
> options	KDTRACE_FRAME
> options	MAC
> options	CAPABILITIES
> options	CAPABILITY_MODE
> options	AUDIT
> options	HWPMC_HOOKS
> options	KBD_INSTALL_CDEV
> options	PRINTF_BUFR_SIZE=128
> options	_KPOSIX_PRIORITY_SCHEDULING
> options	SYSVSEM
> options	SYSVMSG
> options	SYSVSHM
> options	STACK
> options	KTRACE
> options	SCSI_DELAY=5000
> options	COMPAT_FREEBSD7
> options	COMPAT_FREEBSD6
> options	COMPAT_FREEBSD5
> options	COMPAT_FREEBSD4
> options	COMPAT_FREEBSD32
> options	GEOM_LABEL
> options	GEOM_RAID
> options	GEOM_PART_GPT
> options	PSEUDOFS
> options	PROCFS
> options	CD9660
> options	MSDOSFS
> options	NFS_ROOT
> options	NFSLOCKD
> options	NFSD
> options	NFSCL
> options	QUOTA
> options	SCTP
> options	TCP_OFFLOAD
> options	INET6
> options	INET
> options	PREEMPTION
> options	SCHED_ULE
> options	NEW_PCIB
> options	GEOM_PART_MBR
> options	GEOM_PART_EBR_COMPAT
> options	GEOM_PART_EBR
> options	GEOM_PART_BSD
> device	isa
> device	mem
> device	io
> device	uart_ns8250
> device	cpufreq
> device	acpi
> device	pci
> device	fdc
> device	ahci
> device	ata
> device	esp
> device	isci
> device	scbus
> device	ch
> device	da
> device	sa
> device	cd
> device	pass
> device	ses
> device	hptnr
> device	aacraid
> device	atkbdc
> device	atkbd
> device	psm
> device	kbdmux
> device	vga
> device	splash
> device	sc
> device	agp
> device	uart
> device	ppc
> device	ppbus
> device	lpt
> device	ppi
> device	em
> device	miibus
> device	cas
> device	gem
> device	hme
> device	nfe
> device	nge
> device	loop
> device	random
> device	ether
> device	vlan
> device	tun
> device	md
> device	gif
> device	faith
> device	firmware
> device	bpf
> device	uhci
> device	ohci
> device	ehci
> device	xhci
> device	usb
> device	ukbd
> device	umass
> device	virtio
> device	virtio_pci
> device	vtnet
> device	virtio_blk
> device	virtio_scsi
> device	virtio_balloon
> device	netmap
> 
> ------------------------------------------------------------------------
> ddb capture buffer
> 
> ddb: ddb_capture: kvm_nlist
I also notice no SVN  rev in this build nor r254185.  :(

-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c)     E-Mail: ler_at_lerctr.org
US Mail: 108 Turvey Cove, Hutto, TX 78634-5688
Received on Sat Aug 10 2013 - 17:17:53 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:40 UTC