crash with cpucontrol/microcode update : Today's -CURRENT

From: Larry Rosenman <ler_at_lerctr.org>
Date: Sat, 10 Aug 2013 14:06:10 -0500 (CDT)
I'm getting the following _at_R254183:
when I try to run the microcode_update.

Just started with yesterday's -CURRENT.

I can supply the full vmcore and ssh access.


borg.lerctr.org dumped core - see /var/crash/vmcore.7

Sat Aug 10 14:00:00 CDT 2013

FreeBSD borg.lerctr.org 10.0-CURRENT FreeBSD 10.0-CURRENT #16: Sat Aug 10 13:45:06 CDT 2013     root_at_borg.lerctr.org:/usr/obj/usr/src/sys/BORG-DTRACE  amd64

panic: page fault

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
 			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 1378 (cpucontrol)
trap number		= 12
panic: page fault
cpuid = 0
Uptime: 1m48s
Dumping 2337 out of 64479 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%

Reading symbols from /boot/kernel/zfs.ko.symbols...done.
Loaded symbols for /boot/kernel/zfs.ko.symbols
Reading symbols from /boot/kernel/acl_nfs4.ko.symbols...done.
Loaded symbols for /boot/kernel/acl_nfs4.ko.symbols
Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
Loaded symbols for /boot/kernel/opensolaris.ko.symbols
Reading symbols from /boot/kernel/linux.ko.symbols...done.
Loaded symbols for /boot/kernel/linux.ko.symbols
Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done.
Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols
Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done.
Loaded symbols for /boot/kernel/snd_spicds.ko.symbols
Reading symbols from /boot/kernel/sound.ko.symbols...done.
Loaded symbols for /boot/kernel/sound.ko.symbols
Reading symbols from /boot/kernel/coretemp.ko.symbols...done.
Loaded symbols for /boot/kernel/coretemp.ko.symbols
Reading symbols from /boot/kernel/ichsmb.ko.symbols...done.
Loaded symbols for /boot/kernel/ichsmb.ko.symbols
Reading symbols from /boot/kernel/smbus.ko.symbols...done.
Loaded symbols for /boot/kernel/smbus.ko.symbols
Reading symbols from /boot/kernel/ichwd.ko.symbols...done.
Loaded symbols for /boot/kernel/ichwd.ko.symbols
Reading symbols from /boot/kernel/cpuctl.ko.symbols...done.
Loaded symbols for /boot/kernel/cpuctl.ko.symbols
Reading symbols from /boot/kernel/crypto.ko.symbols...done.
Loaded symbols for /boot/kernel/crypto.ko.symbols
Reading symbols from /boot/kernel/cryptodev.ko.symbols...done.
Loaded symbols for /boot/kernel/cryptodev.ko.symbols
Reading symbols from /boot/kernel/dtraceall.ko.symbols...done.
Loaded symbols for /boot/kernel/dtraceall.ko.symbols
Reading symbols from /boot/kernel/profile.ko.symbols...done.
Loaded symbols for /boot/kernel/profile.ko.symbols
Reading symbols from /boot/kernel/cyclic.ko.symbols...done.
Loaded symbols for /boot/kernel/cyclic.ko.symbols
Reading symbols from /boot/kernel/dtrace.ko.symbols...done.
Loaded symbols for /boot/kernel/dtrace.ko.symbols
Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done.
Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols
Reading symbols from /boot/kernel/systrace.ko.symbols...done.
Loaded symbols for /boot/kernel/systrace.ko.symbols
Reading symbols from /boot/kernel/sdt.ko.symbols...done.
Loaded symbols for /boot/kernel/sdt.ko.symbols
Reading symbols from /boot/kernel/lockstat.ko.symbols...done.
Loaded symbols for /boot/kernel/lockstat.ko.symbols
Reading symbols from /boot/kernel/fasttrap.ko.symbols...done.
Loaded symbols for /boot/kernel/fasttrap.ko.symbols
Reading symbols from /boot/kernel/fbt.ko.symbols...done.
Loaded symbols for /boot/kernel/fbt.ko.symbols
Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done.
Loaded symbols for /boot/kernel/dtnfscl.ko.symbols
Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done.
Loaded symbols for /boot/kernel/dtmalloc.ko.symbols
Reading symbols from /boot/kernel/dtio.ko.symbols...done.
Loaded symbols for /boot/kernel/dtio.ko.symbols
Reading symbols from /boot/modules/vboxdrv.ko...done.
Loaded symbols for /boot/modules/vboxdrv.ko
Reading symbols from /boot/modules/nvidia.ko...done.
Loaded symbols for /boot/modules/nvidia.ko
Reading symbols from /boot/kernel/fdescfs.ko.symbols...done.
Loaded symbols for /boot/kernel/fdescfs.ko.symbols
Reading symbols from /boot/kernel/uhid.ko.symbols...done.
Loaded symbols for /boot/kernel/uhid.ko.symbols
Reading symbols from /boot/kernel/ums.ko.symbols...done.
Loaded symbols for /boot/kernel/ums.ko.symbols
Reading symbols from /boot/modules/vboxnetflt.ko...done.
Loaded symbols for /boot/modules/vboxnetflt.ko
Reading symbols from /boot/kernel/netgraph.ko.symbols...done.
Loaded symbols for /boot/kernel/netgraph.ko.symbols
Reading symbols from /boot/kernel/ng_ether.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_ether.ko.symbols
Reading symbols from /boot/modules/vboxnetadp.ko...done.
Loaded symbols for /boot/modules/vboxnetadp.ko
#0  doadump (textdump=<value optimized out>) at pcpu.h:236
236	pcpu.h: No such file or directory.
 	in pcpu.h
(kgdb) #0  doadump (textdump=<value optimized out>) at pcpu.h:236
#1  0xffffffff8051d6f0 in kern_reboot (howto=260)
     at /usr/src/sys/kern/kern_shutdown.c:447
#2  0xffffffff8051da77 in panic (fmt=<value optimized out>)
     at /usr/src/sys/kern/kern_shutdown.c:754
#3  0xffffffff80780d9a in trap_fatal (frame=<value optimized out>,
     eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:873
#4  0xffffffff80781199 in trap_pfault (frame=0x0, usermode=0)
     at /usr/src/sys/amd64/amd64/trap.c:731
#5  0xffffffff80780792 in trap (frame=0xffffff900d55c7b0)
     at /usr/src/sys/amd64/amd64/trap.c:463
#6  0xffffffff8076ad02 in calltrap ()
     at /usr/src/sys/amd64/amd64/exception.S:232
#7  0xffffffff80709be9 in vm_page_unwire (m=0x0, activate=0)
     at /usr/src/sys/vm/vm_page.c:2356
#8  0xffffffff806fb4ed in kmem_unback (object=0xffffffff80c57550,
     addr=<value optimized out>, size=4096) at /usr/src/sys/vm/vm_kern.c:404
#9  0xffffffff806fb5a4 in kmem_free (vmem=0xffffffff80bd7780,
     addr=18446741884987129872, size=4096) at /usr/src/sys/vm/vm_kern.c:421
#10 0xffffffff80506597 in contigfree (addr=0x0, size=4048,
     type=0xffffffff812d5ea0) at /usr/src/sys/kern/kern_malloc.c:435
#11 0xffffffff812d5a79 in cpuctl_ioctl (dev=<value optimized out>,
     cmd=<value optimized out>, data=0xfffffe000d2a2f80 "0?c",
     flags=<value optimized out>, td=<value optimized out>)
     at /usr/src/sys/modules/cpuctl/../../dev/cpuctl/cpuctl.c:480
#12 0xffffffff8041962f in devfs_ioctl_f (fp=0xfffffe001e68cbe0,
     com=3222299396, data=0xfffffe000d2a2f80, cred=<value optimized out>,
     td=0xfffffe0252ebd490) at /usr/src/sys/fs/devfs/devfs_vnops.c:757
#13 0xffffffff8056b3be in kern_ioctl (td=0xfffffe0252ebd490,
     fd=<value optimized out>, com=0) at file.h:306
#14 0xffffffff8056b13f in sys_ioctl (td=0xfffffe0252ebd490,
     uap=0xffffff900d55cb80) at /usr/src/sys/kern/sys_generic.c:693
#15 0xffffffff80781697 in amd64_syscall (td=0xfffffe0252ebd490, traced=0)
     at subr_syscall.c:134
#16 0xffffffff8076afeb in Xfast_syscall ()
     at /usr/src/sys/amd64/amd64/exception.S:391
#17 0x000000080094b4ea in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language:  auto; currently minimal
(kgdb)

------------------------------------------------------------------------
ps -axl

  UID  PID PPID CPU PRI NI    VSZ  RSS MWCHAN   STAT TT     TIME COMMAND
    0    0    0   0  -8  0      0    0 -        DLs   -  0:02.61 [kernel]
    0    1    0   0  52  0   9428    0 wait     DLs   -  0:00.26 [init]
    0    2    0   0 -16  0      0    0 ftcl     DL    -  0:00.00 [ftcleanup]
    0    3    0   0 -16  0      0    0 crypto_w DL    -  0:00.00 [crypto]
    0    4    0   0 -16  0      0    0 crypto_r DL    -  0:00.00 [crypto returns
    0    5    0   0 -16  0      0    0 -        DL    -  0:00.00 [fdc0]
    0    6    0   0  -8  0      0    0 tx->tx_s DL    -  0:00.09 [zfskern]
    0    7    0   0 -16  0      0    0 waiting_ DL    -  0:00.00 [sctp_iterator]
    0    8    0   0 -16  0      0    0 ccb_scan DL    -  0:00.00 [xpt_thrd]
    0    9    0   0 -16  0      0    0 psleep   DL    -  0:00.00 [pagedaemon]
    0   10    0   0 -16  0      0    0 audit_wo DL    -  0:00.00 [audit]
    0   11    0   0 155  0      0    0 -        RL    -  6:08.99 [idle]
    0   12    0   0 -84  0      0    0 -        WL    -  0:00.43 [intr]
    0   13    0   0  -8  0      0    0 -        DL    -  0:00.38 [geom]
    0   14    0   0 -16  0      0    0 -        DL    -  0:00.01 [yarrow]
    0   15    0   0 -68  0      0    0 -        DL    -  0:00.04 [usb]
    0   16    0   0 -20  0      0    0 VBoxIS   DL    -  0:00.00 [TIMER]
    0   17    0   0 -16  0      0    0 psleep   DL    -  0:00.00 [vmdaemon]
    0   18    0   0 155  0      0    0 pgzero   DL    -  0:00.00 [pagezero]
    0   19    0   0 -16  0      0    0 psleep   DL    -  0:00.00 [bufdaemon]
    0   20    0   0  16  0      0    0 syncer   DL    -  0:00.00 [syncer]
    0   21    0   0 -16  0      0    0 vlruwt   DL    -  0:00.00 [vnlru]
    0  847    1   0  52  0  16584    0 select   Ds    -  0:00.00 [moused]
    0  866    1   0  20  0  13208    0 select   Ds    -  0:00.00 [devd]
    0 1004    1   0  20  0  14380    0 -        Rs    -  0:00.05 [syslogd]
    0 1007    1   0 -52  0   6192 2104 nanslp   Ds    -  0:00.00 [watchdogd]
    0 1021    1   0  20  0  16468    0 select   Ds    -  0:00.01 [rpcbind]
    0 1056    1   0  52  0  38964    0 select   Ds    -  0:00.02 [mountd]
    0 1062    1   0  52  0  36804    0 select   Ds    -  0:00.04 [nfsd]
    0 1064 1062   0  52  0  12228    0 rpcsvc   D     -  0:00.00 [nfsd]
    0 1072    1   0  52  0  32716    0 select   Ds    -  0:00.99 [apcupsd]
    0 1103    1   0  20  0  25268    0 select   Ds    -  0:00.01 [ntpd]
    0 1124    0   0 -16  0      0    0 sleep    DL    -  0:00.00 [ng_queue]
    0 1136    1   0  20  0  34748    0 nanslp   Ds    -  0:00.01 [perl]
    0 1140    1   0  52  0  30844    0 nanslp   D     -  0:00.43 [smartd]
   70 1149    1   0  20  0  86688    0 select   Ds    -  0:00.23 [postgres]
   70 1153 1149   0  20  0  86688    0 select   Ds    -  0:00.00 [postgres]
   70 1154 1149   0  20  0  86688    0 select   Ds    -  0:00.00 [postgres]
   70 1155 1149   0  20  0  86688    0 select   Ds    -  0:00.00 [postgres]
   70 1156 1149   0  20  0  46392    0 select   Ds    -  0:00.00 [postgres]
   26 1246    1   0  52  0  48804    0 select   Ds    -  0:00.00 [exim-4.80.1-2]
    0 1255    1   0  20  0 103300    0 kqread   Ds    -  0:00.02 [cupsd]
1028 1261    1   0 155  0  46764    0 select   Ds    -  0:00.07 [boinc_client]
  910 1265    1   0  20  0  65436    0 uwait    Ds    -  0:00.00 [bacula-sd]
    0 1268    1   0  20  0  65260    0 uwait    Ds    -  0:00.00 [bacula-fd]
  910 1271    1   0  20  0  78464    0 uwait    Ds    -  0:00.00 [bacula-dir]
    0 1276    1   0  20  0  56324    0 select   Ds    -  0:00.00 [sshd]
    0 1288    1   0  20  0  16472    0 nanslp   Ds    -  0:00.00 [cron]
    0 1310    1   0  52  0  18592    0 select   Ds    -  0:00.00 [inetd]
    0 1326    1   0  52  0  16944    0 wait     D     -  0:00.00 [sh]
    0 1327    1   0  52  0  12220    0 piperd   D     -  0:00.00 [logger]
    0 1328 1326   0  52  0   8120    0 nanslp   D     -  0:00.00 [sleep]
    0 1330    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
    0 1331    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
    0 1332    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
    0 1333    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
    0 1334    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
    0 1335    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
    0 1336    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.00 [getty]
    0 1337    1   0  52  0  14376    0 ttyin    Ds+   -  0:00.01 [getty]
    0 1339 1276   0  21  0  81592    0 select   Ds    -  0:00.00 [sshd]
1002 1341 1339   0  20  0  81592    0 select   D     -  0:00.00 [sshd]
1002 1342 1341   0  20  0  16944    0 wait     Ds    -  0:00.00 [sh]
    0 1344 1342   0  20  0  51140    0 select   D     -  0:00.00 [sudo]
    0 1345 1344   0  20  0  16944    0 wait     D     -  0:00.00 [sh]
1028 1346 1261   0 155 19  86380    0 -        RN    -  0:00.94 [wcgrid_faah_7.
1028 1347 1261   0 155 19   1892    0 -        RN    -  0:00.04 [wcgrid_cep2_6.
1028 1348 1261   0 155 19  68404    0 -        RN    -  0:01.31 [wcgrid_faah_7.
1028 1349 1261   0 155 19  68404    0 -        RN    -  0:00.74 [wcgrid_faah_7.
1028 1350 1261   0 155 19  90036    0 -        RN    -  0:00.90 [wcgrid_faah_7.
1028 1351 1261   0 155 19  68412    0 -        RN    -  0:00.73 [wcgrid_faah_7.
1028 1352 1261   0 155 19  67892    0 -        RN    -  0:00.88 [wcgrid_faah_7.
1028 1353 1347   0 155 19   1892    0 i        DN    -  0:00.00 [wcgrid_cep2_6.
1028 1354 1261   0 155 19  76092    0 -        RN    -  0:00.80 [wcgrid_faah_7.
1028 1355 1353   0 155 19   1892    0 i        DN    -  0:00.00 [wcgrid_cep2_6.
1028 1356 1261   0 155 19  68404    0 i        DN    -  0:00.00 [wcgrid_faah_7.
1028 1357 1261   0 155 19  68412    0 -        RN    -  0:00.00 [wcgrid_faah_7.
1028 1358 1261   0 155 19  86380    0 -        RN    -  0:00.00 [wcgrid_faah_7.
1028 1359 1261   0 155 19  68404    0 i        DN    -  0:00.00 [wcgrid_faah_7.
1028 1360 1261   0 155 19  67892    0 i        DN    -  0:00.00 [wcgrid_faah_7.
1028 1361 1261   0 155 19  76092    0 i        DN    -  0:00.00 [wcgrid_faah_7.
1028 1362 1261   0 155 19  90036    0 -        RN    -  0:00.00 [wcgrid_faah_7.
    0 1372 1345   0  43  0  16944    0 wait     D+    -  0:00.01 [sh]
    0 1378 1372   0  30  0  12228    0 -        R+    -  0:00.00 [cpucontrol]

------------------------------------------------------------------------
vmstat -s

    319667 cpu context switches
     33520 device interrupts
      4096 software interrupts
    633273 traps
   5813583 system calls
        22 kernel threads created
      1128  fork() calls
       219 vfork() calls
         9 rfork() calls
         0 swap pager pageins
         0 swap pager pages paged in
         0 swap pager pageouts
         0 swap pager pages paged out
      1154 vnode pager pageins
      8819 vnode pager pages paged in
         8 vnode pager pageouts
        16 vnode pager pages paged out
         0 page daemon wakeups
         0 pages examined by the page daemon
        11 pages reactivated
     42644 copy-on-write faults
       255 copy-on-write optimized faults
    562815 zero fill pages zeroed
         0 zero fill pages prezeroed
         5 intransit blocking page faults
    622067 total VM faults taken
      1014 page faults requiring I/O
         0 pages affected by kernel thread creation
     42834 pages affected by  fork()
      8586 pages affected by vfork()
    115675 pages affected by rfork()
         0 pages cached
    823364 pages freed
         0 pages freed by daemon
         0 pages freed by exiting processes
     26936 pages active
      8342 pages inactive
         4 pages in VM cache
    220798 pages wired down
  15823471 pages free
      4096 bytes per page
    240707 total name lookups
           cache hits (90% pos + 4% neg) system 0% per-directory
           deletions 0%, falsehits 0%, toolong 0%

------------------------------------------------------------------------
vmstat -m

          Type InUse MemUse HighUse Requests  Size(s)
       kdtrace   462   100K       -     1922  64,256
          kenv    84    11K       -      112  16,32,64,128
        kqueue     2     3K       -       30  256,2048
     proc-args    53     5K       -      513  16,32,64,128,256
        DEVFSP     3     1K       -       38  64
         hhook     2     1K       -        2  256
       ithread   122    22K       -      122  32,128,256
        KTRACE   100    13K       -      100  128
        linker   401   197K       -      544  16,32,64,128,256,512,1024,2048,4096
         lockf    53     6K       -       89  64,128
    loginclass     1     1K       -        6  64
         cache     1     1K       -        1  32
        devbuf 18685 36135K       -    19791  16,32,64,128,256,512,1024,2048,4096
          temp    57    35K       -     6961  16,32,64,128,256,512,1024,2048,4096
        ip6ndp     5     1K       -        6  64,128
        module   301    38K       -      301  128
      mtx_pool     2    16K       -        2
           osd     7     1K       -       18  16,32,64,128
      pmchooks     1     1K       -        1  128
        NFS fh     1     1K       -        5  16
          pgrp    40     5K       -       62  128
       session    37     5K       -       48  128
          proc     2   256K       -        2
       subproc   194   360K       -     1489  512,4096
          cred   108    17K       -     8666  64,256
        plimit    23     6K       -      204  256
       uidinfo     7    33K       -       25  128
        sysctl     0     0K       -      526  16,32,64
     sysctloid  6788   337K       -     6940  16,32,64,128
     sysctltmp     0     0K       -      384  16,32,64,128,256,4096
       tidhash     1   256K       -        1
       callout     9  3208K       -        9
          umtx   792    99K       -      792  128
      p1003.1b     1     1K       -        1  16
          SWAP    12 19681K       -       12  64
           bus   945    90K       -     5370  16,32,64,128,256,512,1024
        bus-sc   141   293K       -     2331  16,32,64,128,256,512,1024,2048,4096
       devstat    34    69K       -       34  32,4096
  eventhandler    90     8K       -       90  64,128
          kobj   180   720K       -      902  4096
       Per-cpu     1     1K       -        1  32
          rman   360    41K       -      797  16,32,128
          sbuf     0     0K       -     3645  16,32,64,128,256,512,1024,2048,4096
        sglist     3     1K       -        3  32
         stack     0     0K       -        2  256
     taskqueue   109    17K       -      139  16,32,64,256,1024
        Unitno    20     2K       -      420  32,64
          vmem     2   160K       -        4
      ioctlops     1     1K       -     2920  16,32,64,128,256,512,1024,2048
        select    35     5K       -       35  128
           iov     1     1K       -     1252  16,64,128,256,512
           msg     4    30K       -        4  2048,4096
           sem     4   106K       -        4  2048,4096
           shm     6    30K       -       13  2048
           tty    20    20K       -       22  1024,2048
           pts     1     1K       -        1  256
      mbuf_tag     0     0K       -       59  32,128
         shmfd     1     8K       -        1
        soname     7     1K       -    22000  16,32,128
           pcb    37  8341K       -       77  16,32,128,1024,2048
     newnfsmnt     1     1K       -        1  1024
      vfscache     1 16384K       -        1
      vfs_hash     1  8192K       -        1
        vnodes     1     1K       -        1  256
     pfs_nodes    21     6K       -       21  256
   pfs_vncache    86     6K       -       88  64
         mount   396    20K       -     1854  16,32,64,128,256,512
          GEOM   344    60K       -     3318  16,32,64,128,256,512,1024,2048
   vnodemarker     0     0K       -      161  512
           BPF     3     1K       -        3  128
        acpica  1830   187K       -    52091  16,32,64,128,256,512,1024,2048
         ifnet     4     7K       -        4  128,2048
        ifaddr    48    15K       -       48  32,64,128,256,512,2048,4096
   ether_multi    47     3K       -       54  16,32,64
         clone     6     1K       -        6  128
        arpcom     2     1K       -        2  16
       lltable    13     5K       -       13  256,512
          UART     6     5K       -        6  16,1024
       CAM CCB    13    26K       -      101  2048
      acpitask     1     8K       -        1
       acpisem    22     3K       -       22  128
      CAM path    22     1K       -       68  32
       acpidev    36     3K       -       36  64
      routetbl    35     5K       -      228  32,64,128,256,512
          igmp     3     1K       -        3  256
      in_multi     2     1K       -        2  256
    CAM periph    16     4K       -       40  16,32,64,128,256
     CAM queue    31     8K       -      100  16,32,512
CAM dev queue     8     1K       -        8  32
     sctp_a_it     0     0K       -        3  16
      sctp_vrf     1     1K       -        1  64
      sctp_ifa     5     1K       -        5  128
      sctp_ifn     2     1K       -        2  128
     sctp_iter     0     0K       -        3  256
     hostcache     1    28K       -        1
      syncache     1    64K       -        1
   in6_mfilter     1     1K       -        1  1024
     in6_multi    27     4K       -       27  32,256
  ip6_moptions     2     1K       -        2  32,256
     raid_data     0     0K       -      480  32,128,256
           mld     3     1K       -        3  128
       CAM SIM     8     2K       -        8  256
       CAM XPT    58     4K       -      342  16,32,64,128,256,1024
       NFS FHA     1     2K       -        1  2048
           rpc    19     5K       -       23  32,64,128,256,512,1024
audit_evclass   188     6K       -      229  32
     vm_pgdata     7  8193K       -        7  128
       UMAHash     3     5K       -        6  512,1024,2048,4096
       scsi_cd     0     0K       -       11  16
           USB    50   162K       -       56  16,32,64,128,256,512,1024,4096
        USBdev    76    26K       -      112  32,64,128,512,2048,4096
      pci_link    16     2K       -       16  32,64,128
     acpi_perf     8     1K       -        8  64
       memdesc     1     4K       -        1  4096
      atkbddev     2     1K       -        2  64
md_nvidia_data     0     0K       -       78  512
        kbdmux     7    18K       -        7  16,512,1024,2048
           LED     4     1K       -        4  16,128
   md_sii_data     0     0K       -       78  512
       CAM DEV    15    30K       -       24  2048
       ata_pci     1     1K       -        1  64
      acpiintr     1     1K       -        1  64
      ppbusdev     2     1K       -        2  256
        apmdev     1     1K       -        1  128
    madt_table     0     0K       -        1  4096
        isadev     5     1K       -        5  128
random_adaptors     1     1K       -        1  32
       entropy  1024    64K       -     1024  64
        DEVFS3   223    56K       -      269  256
       io_apic     2     4K       -        2  2048
        DEVFS1   194    97K       -      236  512
           MCA     8     1K       -        8  128
          cdev     8     2K       -        8  256
           msi     2     1K       -        2  128
      nexusdev     4     1K       -        4  16
         DEVFS    42     1K       -       43  16,128
      filedesc    86   267K       -     1390  16,32,64,128,2048,4096
filedesc_to_leader    17     2K       -       17  64
         sigio     1     1K       -        1  64
      filecaps     0     0K       -        2  64
         linux    43     3K       -       43  32,64
         mixer     1     4K       -        1  4096
        feeder    16     2K       -       18  32,128
    kstat_data     5     1K       -        5  64
       solaris 64977 432023K       -   918878  16,32,64,128,256,512,1024,2048,4096
        spicds     7     1K       -        7  128
      envy24ht    15   195K       -       15  64,2048
        cpuctl     2     5K       -        2  64,4096
        crypto     1     1K       -        1  512
         xform     0     0K       -      167  16,32
        cyclic    32     3K       -       32  16,64,128
           fbt     1   256K       -        1
      iprtheap    23    56K       -       23  32,64,128,256,2048
        nvidia   179   862K       -      182  16,32,64,128,256,512,1024,2048,4096
   fdesc_mount     1     1K       -        1  16
netgraph_node     4     1K       -        4  128,256
      netgraph     2     1K       -        2  64

------------------------------------------------------------------------
vmstat -z

ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               384,      0,     172,       8,     172,   0,   0
UMA Zones:             1664,      0,     188,       0,     188,   0,   0
UMA Slabs:               80,      0,   22943,     257,   39931,   0,   0
UMA RCntSlabs:           88,      0,     816,      39,     816,   0,   0
UMA Hash:               256,      0,       7,       8,      10,   0,   0
4 Bucket:                32,      0,     252,     748,    5062, 786,   0
8 Bucket:                64,      0,     134,    1230,    2869, 188,   0
16 Bucket:              128,      0,     121,    1150,    2651,  26,   0
32 Bucket:              256,      0,     169,     761,    4898,  92,   0
64 Bucket:              512,      0,     427,     253,    3989, 334,   0
128 Bucket:            1024,      0,    1577,     147,    6403,  23,   0
vmem btag:               56,      0,   15370,     605,   16546, 226,   0
VM OBJECT:              256,      0,    3138,     342,   18576,   0,   0
RADIX NODE:             144,      0,   13651,    1037,   93009,  39,   0
MAP:                    240,      0,       3,      61,       3,   0,   0
KMAP ENTRY:             128,      0,      13,     514,      13,   2,   0
MAP ENTRY:              128,      0,    1705,    1178,   41317,   0,   0
VMSPACE:                408,      0,      53,     181,    1357,   0,   0
fakepg:                 104,      0,       0,       0,       0,   0,   0
mt_zone:               4112,      0,     288,       0,     288,   0,   0
16:                      16,      0,       5,     246,       5,   0,   0
16:                      16,      0,    2467,     545,    2622,   0,   0
16:                      16,      0,       2,     249,       2,   0,   0
16:                      16,      0,      89,     915,   24074,   0,   0
16:                      16,      0,       4,    1000,     254,   0,   0
16:                      16,      0,     651,     855,    4300,   0,   0
16:                      16,      0,      48,     454,      80,   0,   0
16:                      16,      0,    3154,     862,   71341,   0,   0
32:                      32,      0,       2,     123,       2,   0,   0
32:                      32,      0,    2634,     491,    2720,   0,   0
32:                      32,      0,      10,     240,      10,   0,   0
32:                      32,      0,     109,     891,    4525,   0,   0
32:                      32,      0,     250,     750,     649,   0,   0
32:                      32,      0,     337,    1288,    2957,   0,   0
32:                      32,      0,      79,     921,     480,   0,   0
32:                      32,      0,    1963,    2412,   48008,   0,   0
64:                      64,      0,       6,     428,       6,   0,   0
64:                      64,      0,     330,     600,     340,   0,   0
64:                      64,      0,      14,     296,      14,   0,   0
64:                      64,      0,     781,    1017,    2205,   0,   0
64:                      64,      0,     103,     951,    1404,   0,   0
64:                      64,      0,    9185,     735,   16699,   0,   0
64:                      64,      0,    1093,     829,    1093,   0,   0
64:                      64,      0,   25104,    5524,  244655,   0,   0
128:                    128,      0,       8,     519,       8,   0,   0
128:                    128,      0,    2007,    1000,    2513,   0,   0
128:                    128,      0,      45,     978,      67,   0,   0
128:                    128,      0,    1239,     900,   22787,   0,   0
128:                    128,      0,    1164,     603,    1245,   0,   0
128:                    128,      0,    2042,     686,   25479,   0,   0
128:                    128,      0,       3,     276,       3,   0,   0
128:                    128,      0,    9616,    2288,   86497,   0,   0
256:                    256,      0,       2,      73,       2,   0,   0
256:                    256,      0,      34,     461,     241,   0,   0
256:                    256,      0,       6,     129,       6,   0,   0
256:                    256,      0,     242,     403,     902,   0,   0
256:                    256,      0,     465,     300,     950,   0,   0
256:                    256,      0,     397,     338,    5080,   0,   0
256:                    256,      0,       1,      74,       1,   0,   0
256:                    256,      0,    4397,    4423,   98046,   0,   0
512:                    512,      0,       4,      84,       8,   0,   0
512:                    512,      0,      39,      49,     195,   0,   0
512:                    512,      0,       0,     232,     161,   0,   0
512:                    512,      0,     235,     165,    1414,   0,   0
512:                    512,      0,      18,      70,      18,   0,   0
512:                    512,      0,     338,     126,     434,   0,   0
512:                    512,      0,       0,       0,       0,   0,   0
512:                    512,      0,   10447,     601,  209776,   0,   0
1024:                  1024,      0,       1,      15,       1,   0,   0
1024:                  1024,      0,       1,      67,     100,   0,   0
1024:                  1024,      0,       5,      23,       5,   0,   0
1024:                  1024,      0,       6,      62,    1720,   0,   0
1024:                  1024,      0,      20,      20,      20,   0,   0
1024:                  1024,      0,      26,     262,    3723,   0,   0
1024:                  1024,      0,       1,      15,       1,   0,   0
1024:                  1024,      0,     339,      93,   51688,   0,   0
2048:                  2048,      0,       5,      31,      12,   0,   0
2048:                  2048,      0,      32,      20,     129,   0,   0
2048:                  2048,      0,       1,       5,       1,   0,   0
2048:                  2048,      0,      15,      11,     227,   0,   0
2048:                  2048,      0,      76,      32,    1373,   0,   0
2048:                  2048,      0,      25,      33,     203,   0,   0
2048:                  2048,      0,       5,       1,       5,   0,   0
2048:                  2048,      0,     365,      45,    4185,   0,   0
4096:                  4096,      0,       1,       0,       1,   0,   0
4096:                  4096,      0,       2,       0,       2,   0,   0
4096:                  4096,      0,     180,       1,     902,   0,   0
4096:                  4096,      0,      12,       0,      14,   0,   0
4096:                  4096,      0,       5,       2,       8,   0,   0
4096:                  4096,      0,     132,       7,    4551,   0,   0
4096:                  4096,      0,       0,       0,       0,   0,   0
4096:                  4096,      0,    5874,    9408,   89410,   0,   0
uint64 pcpu:              8,      0,    1386,     150,    1386,   0,   0
SLEEPQUEUE:              80,      0,     397,     750,     397,   0,   0
Files:                   80,      0,     207,     793,   14296,   0,   0
TURNSTILE:              136,      0,     397,     283,     397,   0,   0
rl_entry:                40,      0,      49,     951,      49,   0,   0
umtx pi:                 96,      0,       0,       0,       0,   0,   0
MAC labels:              40,      0,       0,       0,       0,   0,   0
PROC:                  1208,      0,      83,      34,    1378,   0,   0
THREAD:                1168,      0,     377,      19,     542,   0,   0
cpuset:                  72,      0,     286,     759,     438,   0,   0
cyclic_id_cache:         64,      0,       0,       0,       0,   0,   0
audit_record:          1240,      0,       0,       0,       0,   0,   0
mbuf_packet:            256, 25727505,    1023,     607,    1407,   0,   0
mbuf:                   256, 25727505,       2,     878,    1060,   0,   0
mbuf_cluster:          2048, 4019922,    1625,       1,    1625,1490,   0
mbuf_jumbo_page:       4096, 2009961,       0,       3,       7,   0,   0
mbuf_jumbo_9k:         9216, 1786632,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384, 1339972,       0,       0,       0,   0,   0
mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   0
dtrace_state_cache:    4096,      0,       0,       0,       0,   0,   0
ttyinq:                 160,      0,     180,     320,     540,   0,   0
ttyoutq:                256,      0,      95,     400,     287,   0,   0
g_bio:                  248,      0,       0,     736,  100826,   0,   0
ata_request:            336,      0,       0,     154,      70,   0,   0
vtnet_tx_hdr:            24,      0,       0,       0,       0,   0,   0
cryptop:                 88,      0,       0,       0,       0,   0,   0
cryptodesc:              72,      0,       0,       0,       0,   0,   0
nv_stack_t:           12288,      0,       2,       2,      12,   0,   0
FPU_save_area:          512,      0,       0,       0,       0,   0,   0
taskq_zone:              48,      0,       0,       0,       0,   0,   0
taskq_zone:              48,      0,       0,     581,      81,   0,   0
VNODE:                  472,      0,    9463,     113,    9663,   0,   0
VNODEPOLL:              112,      0,       0,       0,       0,   0,   0
BUF TRIE:               144,      0,       0,  105948,       0,   0,   0
NAMEI:                 1024,      0,       0,     132,   57320,   0,   0
S VFS Cache:            108,      0,    9378,     527,   11372,   0,   0
STS VFS Cache:          148,      0,       0,       0,       0,   0,   0
L VFS Cache:            328,      0,     151,     173,     205,   0,   0
LTS VFS Cache:          368,      0,       0,       0,       0,   0,   0
NCLNODE:                528,      0,       1,      13,       1,   0,   0
space_seg_cache:         64,      0,    7959,  101099,  238144,   0,   0
zio_cache:              944,      0,       6,   13970,  295331,   0,   0
zio_link_cache:          48,      0,       0,   15770,  278175,   0,   0
sa_cache:                80,      0,    9242,     658,    9413,   0,   0
dnode_t:                856,      0,   10211,      49,   13930,   0,   0
dmu_buf_impl_t:         224,      0,   19047,     299,   24850,   0,   0
arc_buf_hdr_t:          216,      0,   11737,     467,   13936,   0,   0
arc_buf_t:               72,      0,   10586,     689,   16271,   0,   0
zil_lwb_cache:          192,      0,       5,     495,      54,   0,   0
zfs_znode_cache:        368,      0,    9242,     128,    9413,   0,   0
Mountpoints:            816,      0,      30,      35,      30,   0,   0
pipe:                   744,      0,       9,      86,     814,   0,   0
ksiginfo:               112,      0,     131,    1374,    1687,   0,   0
itimer:                 352,      0,       1,      32,       1,   0,   0
KNOTE:                  128,      0,       6,     521,      40,   0,   0
socket:                 680, 2063352,      61,      71,    4016,   0,   0
unpcb:                  240, 2063360,      16,     480,    3676,   0,   0
ipq:                     56, 125670,       0,       0,       0,   0,   0
udp_inpcb:              392, 2063360,      20,     220,     298,   0,   0
udpcb:                   16, 2063471,      20,     984,     298,   0,   0
tcp_inpcb:              392, 2063360,      24,     216,      36,   0,   0
tcpcb:                 1024, 2063352,      24,      80,      36,   0,   0
tcptw:                   72,  27775,       0,     165,       2,   0,   0
syncache:               152,  15366,       0,      78,       1,   0,   0
hostcache:              136,  15370,       0,       0,       0,   0,   0
tcpreass:                40, 251300,       0,       0,       0,   0,   0
sackhole:                32,      0,       0,       0,       0,   0,   0
sctp_ep:               1408, 2063352,       0,       0,       0,   0,   0
sctp_asoc:             2344,  40000,       0,       0,       0,   0,   0
sctp_laddr:              48,  80012,       0,     332,       4,   0,   0
sctp_raddr:             728,  80000,       0,       0,       0,   0,   0
sctp_chunk:             136, 400026,       0,       0,       0,   0,   0
sctp_readq:             104, 400026,       0,       0,       0,   0,   0
sctp_stream_msg_out:    104, 400026,       0,       0,       0,   0,   0
sctp_asconf:             40, 400000,       0,       0,       0,   0,   0
sctp_asconf_ack:         48, 400060,       0,       0,       0,   0,   0
ripcb:                  392, 2063360,       0,       0,       0,   0,   0
rtentry:                200,      0,      17,     243,      17,   0,   0
selfd:                   56,      0,      76,     989,    4050,   0,   0
SWAPMETA:               288, 8039850,       0,       0,       0,   0,   0
NetGraph items:          72,   4123,       0,       0,       0,   0,   0
NetGraph data items:     72,    527,       0,       0,       0,   0,   0


------------------------------------------------------------------------
vmstat -i

interrupt                          total       rate
irq1: atkbd0                           3          0
irq6: fdc0                            22          0
irq14: ata0                          117          0
irq17: uhci0 ehci0                  1145          7
irq19: uhci1 ahci0+                31666        206
cpu0:timer                          7042         46
irq256: em0                          567          3
cpu6:timer                          5638         36
cpu1:timer                          6302         41
cpu3:timer                          7819         51
cpu2:timer                          4971         32
cpu7:timer                          6608         43
cpu4:timer                          5406         35
cpu5:timer                          6416         41
Total                              83722        547

------------------------------------------------------------------------
pstat -T

207/2063348 files
0M/147455M swap space

------------------------------------------------------------------------
pstat -s

Device          512-blocks     Used    Avail Capacity
/dev/gpt/swap0    50331392        0 50331392     0%
/dev/gpt/swap1    50331392        0 50331392     0%
/dev/gpt/swap2    50331392        0 50331392     0%
/dev/gpt/swap3    50331392        0 50331392     0%
/dev/gpt/swap4    50331392        0 50331392     0%
/dev/gpt/swap5    50331392        0 50331392     0%
Total            301988352        0 301988352     0%

------------------------------------------------------------------------
iostat

iostat: kvm_read(_tk_nin): invalid address (0x0)
iostat: disabling TTY statistics
             ada0             ada1             ada2             cpu
   KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
  15.37  36  0.55  15.50  34  0.52  15.58  36  0.55   0  0  2  0 97

------------------------------------------------------------------------
ipcs -a

Message Queues:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME 
m        65536      5432001 --rw------- pgsql    pgsql    pgsql    pgsql               4     41263104         1149         1149 13:54:11 13:55:13 13:54:11

Semaphores:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME 
s        65536      5432001 --rw------- pgsql    pgsql    pgsql    pgsql              17 13:54:11 13:54:11
s        65537      5432002 --rw------- pgsql    pgsql    pgsql    pgsql              17 13:54:11 13:54:11
s        65538      5432003 --rw------- pgsql    pgsql    pgsql    pgsql              17 13:54:11 13:54:11
s        65539      5432004 --rw------- pgsql    pgsql    pgsql    pgsql              17 13:54:11 13:54:11
s        65540      5432005 --rw------- pgsql    pgsql    pgsql    pgsql              17 13:54:11 13:54:11
s        65541      5432006 --rw------- pgsql    pgsql    pgsql    pgsql              17 13:54:11 13:54:11
s        65542      5432007 --rw------- pgsql    pgsql    pgsql    pgsql              17 13:54:19 13:54:11


------------------------------------------------------------------------
ipcs -T

msginfo:
 	msgmax:        16384	(max characters in a message)
 	msgmni:           40	(# of message queues)
 	msgmnb:         2048	(max characters in a message queue)
 	msgtql:           40	(max # of messages in system)
 	msgssz:            8	(size of a message segment)
 	msgseg:         2048	(# of message segments in system)

shminfo:
 	shmmax:    536870912	(max shared memory segment size)
 	shmmin:            1	(min shared memory segment size)
 	shmmni:          192	(max number of shared memory identifiers)
 	shmseg:          128	(max shared memory segments per process)
 	shmall:       131072	(max amount of shared memory in pages)

seminfo:
 	semmni:           50	(# of semaphore identifiers)
 	semmns:          340	(# of semaphores in system)
 	semmnu:          150	(# of undo structures in system)
 	semmsl:          340	(max # of semaphores per id)
 	semopm:          100	(max # of operations per semop call)
 	semume:           50	(max # of undo entries per process)
 	semusz:          632	(size in bytes of undo structure)
 	semvmx:        32767	(semaphore maximum value)
 	semaem:        16384	(adjust on exit max value)


------------------------------------------------------------------------
nfsstat

Client Info:
Rpc Counts:
   Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
         2         0         0         0         0         0         0         0
    Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
         0         0         0         0         0         0         0         0
     Mknod    Fsstat    Fsinfo  PathConf    Commit
         0         1         1         0         0
Rpc Info:
  TimedOut   Invalid X Replies   Retries  Requests
         0         0         0         0         4
Cache Info:
Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW Hits    Misses
         0         0         0         0         0         0         0         0
BioRLHits    Misses BioD Hits    Misses DirE Hits    Misses Accs Hits    Misses
         0         0         0         0         0         0         0         0

Server Info:
   Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
         0         0         0         0         0         0         0         0
    Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
         0         0         0         0         0         0         0         0
     Mknod    Fsstat    Fsinfo  PathConf    Commit
         0         0         0         0         0
Server Ret-Failed
                 0
Server Faults
             0
Server Cache Stats:
    Inprog      Idem  Non-idem    Misses
         0         0         0         0
Server Write Gathering:
  WriteOps  WriteRPC   Opsaved
         0         0         0

------------------------------------------------------------------------
netstat -s

tcp:
 	167 packets sent
 		138 data packets (12023 bytes)
 		0 data packets (0 bytes) retransmitted
 		0 data packets unnecessarily retransmitted
 		0 resends initiated by MTU discovery
 		21 ack-only packets (4 delayed)
 		0 URG only packets
 		0 window probe packets
 		0 window update packets
 		8 control packets
 	196 packets received
 		137 acks (for 11879 bytes)
 		1 duplicate ack
 		0 acks for unsent data
 		136 packets (16234 bytes) received in-sequence
 		0 completely duplicate packets (0 bytes)
 		0 old duplicate packets
 		0 packets with some dup. data (0 bytes duped)
 		0 out-of-order packets (0 bytes)
 		0 packets (0 bytes) of data after window
 		0 window probes
 		0 window update packets
 		0 packets received after close
 		0 discarded for bad checksums
 		0 discarded for bad header offset fields
 		0 discarded because packet too short
 		0 discarded due to memory problems
 	5 connection requests
 	1 connection accept
 	0 bad connection attempts
 	0 listen queue overflows
 	0 ignored RSTs in the windows
 	6 connections established (including accepts)
 	12 connections closed (including 0 drops)
 		0 connections updated cached RTT on close
 		0 connections updated cached RTT variance on close
 		0 connections updated cached ssthresh on close
 	0 embryonic connections dropped
 	133 segments updated rtt (of 134 attempts)
 	0 retransmit timeouts
 		0 connections dropped by rexmit timeout
 	0 persist timeouts
 		0 connections dropped by persist timeout
 	0 Connections (fin_wait_2) dropped because of timeout
 	0 keepalive timeouts
 		0 keepalive probes sent
 		0 connections dropped by keepalive
 	3 correct ACK header predictions
 	57 correct data packet header predictions
 	1 syncache entry added
 		0 retransmitted
 		0 dupsyn
 		0 dropped
 		1 completed
 		0 bucket overflow
 		0 cache overflow
 		0 reset
 		0 stale
 		0 aborted
 		0 badack
 		0 unreach
 		0 zone failures
 	1 cookie sent
 	0 cookies received
 	0 hostcache entries added
 		0 bucket overflow
 	0 SACK recovery episodes
 	0 segment rexmits in SACK recovery episodes
 	0 byte rexmits in SACK recovery episodes
 	0 SACK options (SACK blocks) received
 	0 SACK options (SACK blocks) sent
 	0 SACK scoreboard overflow
 	0 packets with ECN CE bit set
 	0 packets with ECN ECT(0) bit set
 	0 packets with ECN ECT(1) bit set
 	0 successful ECN handshakes
 	0 times ECN reduced the congestion window
udp:
 	118 datagrams received
 	0 with incomplete header
 	0 with bad data length field
 	0 with bad checksum
 	0 with no checksum
 	0 dropped due to no socket
 	36 broadcast/multicast datagrams undelivered
 	0 dropped due to full socket buffers
 	0 not for hashed pcb
 	82 delivered
 	82 datagrams output
 	0 times multicast source filter matched
ip:
 	267 total packets received
 	0 bad header checksums
 	0 with size smaller than minimum
 	0 with data size < data length
 	0 with ip length > max ip packet size
 	0 with header length < data size
 	0 with data length < header length
 	0 with bad options
 	0 with incorrect version number
 	0 fragments received
 	0 fragments dropped (dup or out of space)
 	0 fragments dropped after timeout
 	0 packets reassembled ok
 	264 packets for this host
 	3 packets for unknown/unsupported protocol
 	0 packets forwarded (0 packets fast forwarded)
 	0 packets not forwardable
 	0 packets received for unknown multicast group
 	0 redirects sent
 	199 packets sent from this host
 	0 packets sent with fabricated ip header
 	0 output packets dropped due to no bufs, etc.
 	0 output packets discarded due to no route
 	0 output datagrams fragmented
 	0 fragments created
 	0 datagrams that can't be fragmented
 	0 tunneling packets that can't find gif
 	0 datagrams with bad address in header
icmp:
 	0 calls to icmp_error
 	0 errors not generated in response to an icmp message
 	0 messages with bad code fields
 	0 messages less than the minimum length
 	0 messages with bad checksum
 	0 messages with bad length
 	0 multicast echo requests ignored
 	0 multicast timestamp requests ignored
 	0 message responses generated
 	0 invalid return addresses
 	0 no return routes
igmp:
 	3 messages received
 	0 messages received with too few bytes
 	0 messages received with wrong TTL
 	0 messages received with bad checksum
 	1 V1/V2 membership query received
 	0 V3 membership queries received
 	0 membership queries received with invalid field(s)
 	1 general query received
 	0 group queries received
 	0 group-source queries received
 	0 group-source queries dropped
 	1 membership report received
 	0 membership reports received with invalid field(s)
 	0 membership reports received for groups to which we belong
 	0 V3 reports received without Router Alert
 	0 membership reports sent
arp:
 	3 ARP requests sent
 	3 ARP replies sent
 	26 ARP requests received
 	7 ARP replies received
 	33 ARP packets received
 	0 total packets dropped due to no ARP entry
 	0 ARP entrys timed out
 	0 Duplicate IPs seen
ip6:
 	50 total packets received
 	0 with size smaller than minimum
 	0 with data size < data length
 	0 with bad options
 	0 with incorrect version number
 	0 fragments received
 	0 fragments dropped (dup or out of space)
 	0 fragments dropped after timeout
 	0 fragments that exceeded limit
 	0 packets reassembled ok
 	50 packets for this host
 	0 packets forwarded
 	0 packets not forwardable
 	0 redirects sent
 	57 packets sent from this host
 	0 packets sent with fabricated ip header
 	0 output packets dropped due to no bufs, etc.
 	4 output packets discarded due to no route
 	0 output datagrams fragmented
 	0 fragments created
 	0 datagrams that can't be fragmented
 	0 packets that violated scope rules
 	0 multicast packets which we don't join
 	Input histogram:
 		UDP: 50
 	Mbuf statistics:
 		26 one mbuf
 		24 one ext mbuf
 		0 two or more ext mbuf
 	0 packets whose headers are not contiguous
 	0 tunneling packets that can't find gif
 	0 packets discarded because of too many headers
 	0 failures of source address selection
 	Source addresses selection rule applied:
icmp6:
 	0 calls to icmp6_error
 	0 errors not generated in response to an icmp6 message
 	0 errors not generated because of rate limitation
 	Output histogram:
 		neighbor solicitation: 1
 	0 messages with bad code fields
 	0 messages < minimum length
 	0 bad checksums
 	0 messages with bad length
 	Histogram of error messages to be generated:
 		0 no route
 		0 administratively prohibited
 		0 beyond scope
 		0 address unreachable
 		0 port unreachable
 		0 packet too big
 		0 time exceed transit
 		0 time exceed reassembly
 		0 erroneous header field
 		0 unrecognized next header
 		0 unrecognized option
 		0 redirect
 		0 unknown
 	0 message responses generated
 	0 messages with too many ND options
 	0 messages with bad ND options
 	0 bad neighbor solicitation messages
 	0 bad neighbor advertisement messages
 	0 bad router solicitation messages
 	0 bad router advertisement messages
 	0 bad redirect messages
 	0 path MTU changes
rip6:
 	0 messages received
 	0 checksum calculations on inbound
 	0 messages with bad checksum
 	0 messages dropped due to no socket
 	0 multicast messages dropped due to no socket
 	0 messages dropped due to full socket buffers
 	0 delivered
 	0 datagrams output

------------------------------------------------------------------------
netstat -m

1025/1485/2510 mbufs in use (current/cache/total)
1018/608/1626/4019922 mbuf clusters in use (current/cache/total/max)
1023/607 mbuf+clusters out of packet secondary zone in use (current/cache)
0/3/3/2009961 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/1786632 9k jumbo clusters in use (current/cache/total/max)
0/0/0/1339972 16k jumbo clusters in use (current/cache/total/max)
2292K/1599K/3891K bytes allocated to network (current/cache/total)
0/1490/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile

------------------------------------------------------------------------
netstat -id

Name    Mtu Network       Address              Ipkts Ierrs Idrop    Opkts Oerrs  Coll Drop
em0    1500 <Link#1>      00:30:48:f2:29:9c      300     0     0      205     0     0    0 
em0    1500 192.168.200.0 borg                   245     -     -      199     -     -    - 
em0    1500 fe80::230:48f fe80::230:48ff:fe        0     -     -        3     -     -    - 
em1*   1500 <Link#2>      00:30:48:f2:29:9d        0     0     0        0     0     0    0 
lo0   16384 <Link#3>                              50     0     0       50     0     0    0 
lo0   16384 localhost     ::1                     50     -     -       50     -     -    - 
lo0   16384 fe80::1%lo0   fe80::1                  0     -     -        0     -     -    - 
lo0   16384 your-net      localhost                0     -     -        0     -     -    -

------------------------------------------------------------------------
netstat -anr

Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.200.11     UGS         0       39    em0
127.0.0.1          link#3             UH          0        0    lo0
192.168.200.0/24   link#1             U           0      160    em0
192.168.200.4      link#1             UHS         0        0    lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#3                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%em0/64                     link#1                        U           em0
fe80::230:48ff:fef2:299c%em0      link#1                        UHS         lo0
fe80::%lo0/64                     link#3                        U           lo0
fe80::1%lo0                       link#3                        UHS         lo0
ff01::%em0/32                     fe80::230:48ff:fef2:299c%em0  U           em0
ff01::%lo0/32                     ::1                           U           lo0
ff02::/16                         ::1                           UGRS        lo0
ff02::%em0/32                     fe80::230:48ff:fef2:299c%em0  U           em0
ff02::%lo0/32                     ::1                           U           lo0

------------------------------------------------------------------------
netstat -anA

Active Internet connections (including servers)
Tcpcb            Proto Recv-Q Send-Q Local Address      Foreign Address    (state)
fffffe025f039c00 tcp4       0      0 192.168.200.4.3351 198.20.8.246.80    ESTABLISHED
fffffe0206042400 tcp4       0      0 127.0.0.1.31416    *.*                LISTEN
fffffe001eff9400 tcp4       0    148 192.168.200.4.22   192.168.200.87.498 ESTABLISHED
fffffe020602e400 tcp4       0      0 *.9101             *.*                LISTEN
fffffe020615e800 tcp4       0      0 *.22               *.*                LISTEN
fffffe020615ec00 tcp6       0      0 *.22               *.*                LISTEN
fffffe020602e800 tcp4       0      0 *.9102             *.*                LISTEN
fffffe023e226400 tcp4       0      0 *.9103             *.*                LISTEN
fffffe020615f000 tcp4       0      0 *.631              *.*                LISTEN
fffffe020615f400 tcp6       0      0 *.631              *.*                LISTEN
fffffe001eff9800 tcp4       0      0 127.0.0.1.587      *.*                LISTEN
fffffe001eff9c00 tcp4       0      0 127.0.0.1.25       *.*                LISTEN
fffffe001effa000 tcp4       0      0 192.168.200.4.587  *.*                LISTEN
fffffe001effa400 tcp4       0      0 192.168.200.4.25   *.*                LISTEN
fffffe001effa800 tcp4       0      0 127.0.0.1.5432     *.*                LISTEN
fffffe001effac00 tcp6       0      0 ::1.5432           *.*                LISTEN
fffffe02063a6000 tcp4       0      0 *.3551             *.*                LISTEN
fffffe020602ec00 tcp6       0      0 *.2049             *.*                LISTEN
fffffe020602f000 tcp4       0      0 *.2049             *.*                LISTEN
fffffe020615f800 tcp4       0      0 *.977              *.*                LISTEN
fffffe020615fc00 tcp6       0      0 *.977              *.*                LISTEN
fffffe001effb000 tcp4       0      0 *.111              *.*                LISTEN
fffffe020602f400 tcp6       0      0 *.111              *.*                LISTEN
fffffe0206042800 tcp4       0      0 192.168.200.4.755  192.168.200.23.204 ESTABLISHED
fffffe001eb7f7a8 udp4       0      0 *.631              *.* 
fffffe001eba77a8 udp6       0      0 ::1.46935          ::1.46935 
fffffe001eb7f930 udp4       0      0 127.0.0.1.123      *.* 
fffffe001eb7fab8 udp6       0      0 fe80:3::1.123      *.* 
fffffe001eb7fc40 udp6       0      0 ::1.123            *.* 
fffffe001eb7fdc8 udp6       0      0 fe80:1::230:48ff.1 *.* 
fffffe001eb04000 udp4       0      0 192.168.200.4.123  *.* 
fffffe001eb04188 udp6       0      0 *.123              *.* 
fffffe001eb04310 udp4       0      0 *.123              *.* 
fffffe001ee6e188 udp6       0      0 *.2049             *.* 
fffffe001ee6e310 udp4       0      0 *.2049             *.* 
fffffe001eb04498 udp4       0      0 *.977              *.* 
fffffe001eb04620 udp6       0      0 *.977              *.* 
fffffe001ee7c188 udp6       0      0 *.*                *.* 
fffffe001eb047a8 udp4       0      0 *.868              *.* 
fffffe001eb04930 udp4       0      0 *.111              *.* 
fffffe001eb04ab8 udp6       0      0 *.1013             *.* 
fffffe001eb04c40 udp6       0      0 *.111              *.* 
fffffe001eb91620 udp4       0      0 *.514              *.* 
fffffe001eb917a8 udp6       0      0 *.514              *.* 
Active UNIX domain sockets
Address  Type   Recv-Q Send-Q    Inode     Conn     Refs  Nextref Addr
fffffe0206108780 stream      0      0        0 fffffe001ee823c0        0        0
fffffe001ee823c0 stream      0      0        0 fffffe0206108780        0        0
fffffe0206108960 stream      0      0 fffffe0206b91588        0        0        0 /var/run/cups.sock
fffffe001ee5b000 stream      0      0 fffffe02068f0760        0        0        0 /tmp/.s.PGSQL.5432
fffffe0206109690 stream      0      0 fffffe020610d760        0        0        0 /var/run/rpcbind.sock
fffffe001ee825a0 stream      0      0 fffffe001e71f938        0        0        0 /var/run/devd.pipe
fffffe001ee822d0 dgram       0      0        0        0        0        0
fffffe001ee824b0 dgram       0      0        0 fffffe02060dd960        0 fffffe001ee89000
fffffe001ee89000 dgram       0      0        0 fffffe02060dd960        0 fffffe0206108870
fffffe0206108870 dgram       0      0        0 fffffe02060dd960        0        0
fffffe02060dd780 dgram       0      0        0 fffffe02060dd870        0 fffffe001ee5b870
fffffe001ee5b870 dgram       0      0        0 fffffe02060dd870        0 fffffe001eeb02d0
fffffe001eeb02d0 dgram       0      0        0 fffffe02060dd870        0 fffffe001ee61b40
fffffe001ee61b40 dgram       0      0        0 fffffe02060dd870        0        0
fffffe02060dd870 dgram       0      0 fffffe001e786938        0 fffffe02060dd780        0 /var/run/logpriv
fffffe02060dd960 dgram       0      0 fffffe001e786b10        0 fffffe001ee824b0        0 /var/run/log

------------------------------------------------------------------------
netstat -aL

Current listen queue sizes (qlen/incqlen/maxqlen)
Proto Listen         Local Address 
tcp4  0/0/128        localhost.31416 
tcp4  0/0/50         *.bacula-dir 
tcp4  0/0/128        *.ssh 
tcp6  0/0/128        *.ssh 
tcp4  0/0/50         *.bacula-fd 
tcp4  0/0/50         *.bacula-sd 
tcp4  0/0/128        *.ipp 
tcp6  0/0/128        *.ipp 
tcp4  0/0/20         localhost.submission 
tcp4  0/0/20         localhost.smtp 
tcp4  0/0/20         borg.submission 
tcp4  0/0/20         borg.smtp 
tcp4  0/0/128        localhost.postgresql 
tcp6  0/0/128        localhost.postgresql 
tcp4  0/0/5          *.3551 
tcp6  0/0/5          *.nfsd 
tcp4  0/0/5          *.nfsd 
tcp4  0/0/128        *.977 
tcp6  0/0/128        *.977 
tcp4  0/0/128        *.sunrpc 
tcp6  0/0/128        *.sunrpc 
unix  0/0/128        /var/run/cups.sock
unix  0/0/128        /tmp/.s.PGSQL.5432
unix  0/0/128        /var/run/rpcbind.sock
unix  0/0/4          /var/run/devd.pipe

------------------------------------------------------------------------
fstat

fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read file 10 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read file 10 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read file 10 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read file 10 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read file 10 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read file 10 at 0xffff
fstat: can't read file 13 at 0xfffffffffffffff
fstat: can't read file 14 at 0x78
fstat: can't read file 16 at 0xffff
fstat: can't read file 19 at 0xfffffffffffffff
fstat: can't read file 20 at 0x78
fstat: can't read file 22 at 0xffff
fstat: can't read file 23 at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read file 10 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read file 7 at 0xfffffffffffffff
fstat: can't read file 8 at 0x78
fstat: can't read file 10 at 0xffff
fstat: can't read file 13 at 0xfffffffffffffff
fstat: can't read file 14 at 0x78
fstat: can't read file 16 at 0xffff
fstat: can't read file 19 at 0xfffffffffffffff
fstat: can't read file 20 at 0x78
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0xfffffffffffffff
fstat: can't read file 2 at 0x78
fstat: can't read file 4 at 0xffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
root     cpucontrol  1378 root -         -       error    -
root     cpucontrol  1378   wd -         -       error    -
root     cpucontrol  1378 text -         -       error    -
root     cpucontrol  1378 ctty /dev        169 crw--w----   pts/0 rw
root     cpucontrol  1378    0 /dev        169 crw--w----   pts/0 rw
root     sh          1372 root -         -       error    -
root     sh          1372   wd -         -       error    -
root     sh          1372 text -         -       error    -
root     sh          1372 ctty /dev        169 crw--w----   pts/0 rw
root     sh          1372    0 /dev        169 crw--w----   pts/0 rw
root     sh          1372    6 /dev        169 crw--w----   pts/0 rw
boinc    wcgrid_faah_7.15_i  1362 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1362   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1362 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1362    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1362    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1361 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1361   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1361 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1361    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1361    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1360 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1360   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1360 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1360    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1360    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1359 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1359   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1359 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1359    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1359    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1358 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1358   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1358 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1358    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1358    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1357 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1357   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1357 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1357    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1357    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1356 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1356   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1356 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1356    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1356    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_cep2_6.40_i  1355 root -         -       error    -
boinc    wcgrid_cep2_6.40_i  1355   wd -         -       error    -
boinc    wcgrid_cep2_6.40_i  1355 text -         -       error    -
boinc    wcgrid_cep2_6.40_i  1355    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_cep2_6.40_i  1355    6 -         -       error    -
boinc    wcgrid_faah_7.15_i  1354 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1354   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1354 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1354    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1354    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_cep2_6.40_i  1353 root -         -       error    -
boinc    wcgrid_cep2_6.40_i  1353   wd -         -       error    -
boinc    wcgrid_cep2_6.40_i  1353 text -         -       error    -
boinc    wcgrid_cep2_6.40_i  1353    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_cep2_6.40_i  1353    6 -         -       error    -
boinc    wcgrid_faah_7.15_i  1352 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1352   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1352 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1352    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1352    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1351 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1351   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1351 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1351    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1351    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1350 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1350   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1350 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1350    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1350    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1349 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1349   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1349 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1349    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1349    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1348 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1348   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1348 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1348    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1348    6 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_cep2_6.40_i  1347 root -         -       error    -
boinc    wcgrid_cep2_6.40_i  1347   wd -         -       error    -
boinc    wcgrid_cep2_6.40_i  1347 text -         -       error    -
boinc    wcgrid_cep2_6.40_i  1347    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_cep2_6.40_i  1347    6 -         -       error    -
boinc    wcgrid_faah_7.15_i  1346 root -         -       error    -
boinc    wcgrid_faah_7.15_i  1346   wd -         -       error    -
boinc    wcgrid_faah_7.15_i  1346 text -         -       error    -
boinc    wcgrid_faah_7.15_i  1346    0 /dev         30 crw-rw-rw-    null rw
boinc    wcgrid_faah_7.15_i  1346    6 /dev         30 crw-rw-rw-    null rw
root     sh          1345 root -         -       error    -
root     sh          1345   wd -         -       error    -
root     sh          1345 text -         -       error    -
root     sh          1345 ctty /dev        169 crw--w----   pts/0 rw
root     sh          1345    0 /dev        169 crw--w----   pts/0 rw
root     sh          1345    6 /dev        169 crw--w----   pts/0 rw
root     sudo        1344 root -         -       error    -
root     sudo        1344   wd -         -       error    -
root     sudo        1344 text -         -       error    -
root     sudo        1344 ctty /dev        169 crw--w----   pts/0 rw
root     sudo        1344    0 /dev        169 crw--w----   pts/0 rw
ler      sh          1342 root -         -       error    -
ler      sh          1342   wd -         -       error    -
ler      sh          1342 text -         -       error    -
ler      sh          1342 ctty /dev        169 crw--w----   pts/0 rw
ler      sh          1342    0 /dev        169 crw--w----   pts/0 rw
ler      sh          1342    6 /dev        169 crw--w----   pts/0 rw
ler      sshd        1341 root -         -       error    -
ler      sshd        1341   wd -         -       error    -
ler      sshd        1341 text -         -       error    -
ler      sshd        1341    0 /dev         30 crw-rw-rw-    null rw
ler      sshd        1341    6 /dev         30 crw-rw-rw-    null rw
root     sshd        1339 root -         -       error    -
root     sshd        1339   wd -         -       error    -
root     sshd        1339 text -         -       error    -
root     sshd        1339    0 /dev         30 crw-rw-rw-    null  r
root     sshd        1339    6 /dev         30 crw-rw-rw-    null rw
root     getty       1337 root -         -       error    -
root     getty       1337   wd -         -       error    -
root     getty       1337 text -         -       error    -
root     getty       1337 ctty /dev         83 crw-------   ttyv7 rw
root     getty       1337    0 /dev         83 crw-------   ttyv7 rw
root     getty       1336 root -         -       error    -
root     getty       1336   wd -         -       error    -
root     getty       1336 text -         -       error    -
root     getty       1336 ctty /dev         82 crw-------   ttyv6 rw
root     getty       1336    0 /dev         82 crw-------   ttyv6 rw
root     getty       1335 root -         -       error    -
root     getty       1335   wd -         -       error    -
root     getty       1335 text -         -       error    -
root     getty       1335 ctty /dev         81 crw-------   ttyv5 rw
root     getty       1335    0 /dev         81 crw-------   ttyv5 rw
root     getty       1334 root -         -       error    -
root     getty       1334   wd -         -       error    -
root     getty       1334 text -         -       error    -
root     getty       1334 ctty /dev         80 crw-------   ttyv4 rw
root     getty       1334    0 /dev         80 crw-------   ttyv4 rw
root     getty       1333 root -         -       error    -
root     getty       1333   wd -         -       error    -
root     getty       1333 text -         -       error    -
root     getty       1333 ctty /dev         79 crw-------   ttyv3 rw
root     getty       1333    0 /dev         79 crw-------   ttyv3 rw
root     getty       1332 root -         -       error    -
root     getty       1332   wd -         -       error    -
root     getty       1332 text -         -       error    -
root     getty       1332 ctty /dev         78 crw-------   ttyv2 rw
root     getty       1332    0 /dev         78 crw-------   ttyv2 rw
root     getty       1331 root -         -       error    -
root     getty       1331   wd -         -       error    -
root     getty       1331 text -         -       error    -
root     getty       1331 ctty /dev         77 crw-------   ttyv1 rw
root     getty       1331    0 /dev         77 crw-------   ttyv1 rw
root     getty       1330 root -         -       error    -
root     getty       1330   wd -         -       error    -
root     getty       1330 text -         -       error    -
root     getty       1330 ctty /dev         76 crw-------   ttyv0 rw
root     getty       1330    0 /dev         76 crw-------   ttyv0 rw
root     sleep       1328 root -         -       error    -
root     sleep       1328   wd -         -       error    -
root     sleep       1328 text -         -       error    -
root     sleep       1328    0 /dev         30 crw-rw-rw-    null  r
root     logger      1327 root -         -       error    -
root     logger      1327   wd -         -       error    -
root     logger      1327 text -         -       error    -
root     logger      1327    0* pipe fffffe001e7bfba0 <-> fffffe001e7bfd00      0 rw
root     sh          1326 root -         -       error    -
root     sh          1326   wd -         -       error    -
root     sh          1326 text -         -       error    -
root     sh          1326    0 /dev         30 crw-rw-rw-    null  r
root     inetd       1310 root -         -       error    -
root     inetd       1310   wd -         -       error    -
root     inetd       1310 text -         -       error    -
root     inetd       1310    0 /dev         30 crw-rw-rw-    null rw
root     cron        1288 root -         -       error    -
root     cron        1288   wd -         -       error    -
root     cron        1288 text -         -       error    -
root     cron        1288    0 /dev         30 crw-rw-rw-    null rw
root     sshd        1276 root -         -       error    -
root     sshd        1276   wd -         -       error    -
root     sshd        1276 text -         -       error    -
root     sshd        1276    0 /dev         30 crw-rw-rw-    null rw
bacula   bacula-dir  1271 root -         -       error    -
bacula   bacula-dir  1271   wd -         -       error    -
bacula   bacula-dir  1271 text -         -       error    -
bacula   bacula-dir  1271    0 /dev         30 crw-rw-rw-    null  r
root     bacula-fd   1268 root -         -       error    -
root     bacula-fd   1268   wd -         -       error    -
root     bacula-fd   1268 text -         -       error    -
root     bacula-fd   1268    0 /dev         30 crw-rw-rw-    null  r
bacula   bacula-sd   1265 root -         -       error    -
bacula   bacula-sd   1265   wd -         -       error    -
bacula   bacula-sd   1265 text -         -       error    -
bacula   bacula-sd   1265    0 /dev         30 crw-rw-rw-    null  r
boinc    boinc_client  1261 root -         -       error    -
boinc    boinc_client  1261   wd -         -       error    -
boinc    boinc_client  1261 text -         -       error    -
boinc    boinc_client  1261    0 /dev         30 crw-rw-rw-    null rw
boinc    boinc_client  1261    6 -         -       error    -
root     cupsd       1255 root -         -       error    -
root     cupsd       1255   wd -         -       error    -
root     cupsd       1255 text -         -       error    -
root     cupsd       1255    0 /dev         30 crw-rw-rw-    null  r
root     cupsd       1255    6 /dev         30 crw-rw-rw-    null  w
root     cupsd       1255   12 /dev         30 crw-rw-rw-    null  w
mailnull exim-4.80.1-2  1246 root -         -       error    -
mailnull exim-4.80.1-2  1246   wd -         -       error    -
mailnull exim-4.80.1-2  1246 text -         -       error    -
mailnull exim-4.80.1-2  1246    0 /dev         30 crw-rw-rw-    null rw
mailnull exim-4.80.1-2  1246    6 /dev         30 crw-rw-rw-    null rw
pgsql    postgres    1156 root -         -       error    -
pgsql    postgres    1156   wd -         -       error    -
pgsql    postgres    1156 text -         -       error    -
pgsql    postgres    1156    0 /dev         30 crw-rw-rw-    null  r
pgsql    postgres    1156    6 -         -       error    -
pgsql    postgres    1155 root -         -       error    -
pgsql    postgres    1155   wd -         -       error    -
pgsql    postgres    1155 text -         -       error    -
pgsql    postgres    1155    0 /dev         30 crw-rw-rw-    null  r
pgsql    postgres    1155    6 -         -       error    -
pgsql    postgres    1154 root -         -       error    -
pgsql    postgres    1154   wd -         -       error    -
pgsql    postgres    1154 text -         -       error    -
pgsql    postgres    1154    0 /dev         30 crw-rw-rw-    null  r
pgsql    postgres    1154    6 -         -       error    -
pgsql    postgres    1153 root -         -       error    -
pgsql    postgres    1153   wd -         -       error    -
pgsql    postgres    1153 text -         -       error    -
pgsql    postgres    1153    0 /dev         30 crw-rw-rw-    null  r
pgsql    postgres    1153    6 -         -       error    -
pgsql    postgres    1149 root -         -       error    -
pgsql    postgres    1149   wd -         -       error    -
pgsql    postgres    1149 text -         -       error    -
pgsql    postgres    1149    0 /dev         30 crw-rw-rw-    null  r
pgsql    postgres    1149    6 -         -       error    -
root     smartd      1140 root -         -       error    -
root     smartd      1140   wd -         -       error    -
root     smartd      1140 text -         -       error    -
root     smartd      1140    0 /dev         30 crw-rw-rw-    null rw
root     perl        1136 root -         -       error    -
root     perl        1136   wd -         -       error    -
root     perl        1136 text -         -       error    -
root     perl        1136    0 -         -       error    -
root     ng_queue    1124 root -         -       error    -
root     ng_queue    1124   wd -         -       error    -
root     ntpd        1103 root -         -       error    -
root     ntpd        1103   wd -         -       error    -
root     ntpd        1103 text -         -       error    -
root     ntpd        1103    0 /dev         30 crw-rw-rw-    null rw
root     ntpd        1103    6 /dev         30 crw-rw-rw-    null rw
root     ntpd        1103   12 /dev         30 crw-rw-rw-    null rw
root     ntpd        1103   18* local dgram fffffe001eeb02d0 <-> fffffe02060dd870
root     apcupsd     1072 root -         -       error    -
root     apcupsd     1072   wd -         -       error    -
root     apcupsd     1072 text -         -       error    -
root     apcupsd     1072    0 /dev         30 crw-rw-rw-    null  r
root     apcupsd     1072    6 /dev         30 crw-rw-rw-    null  r
root     nfsd        1064 root -         -       error    -
root     nfsd        1064   wd -         -       error    -
root     nfsd        1064 text -         -       error    -
root     nfsd        1064    0 /dev         30 crw-rw-rw-    null rw
root     nfsd        1062 root -         -       error    -
root     nfsd        1062   wd -         -       error    -
root     nfsd        1062 text -         -       error    -
root     nfsd        1062    0 /dev         30 crw-rw-rw-    null rw
root     nfsd        1062    6 /dev         30 crw-rw-rw-    null rw
root     mountd      1056 root -         -       error    -
root     mountd      1056   wd -         -       error    -
root     mountd      1056 text -         -       error    -
root     mountd      1056    0 /dev         30 crw-rw-rw-    null rw
root     mountd      1056    6 /dev         30 crw-rw-rw-    null rw
root     rpcbind     1021 root -         -       error    -
root     rpcbind     1021   wd -         -       error    -
root     rpcbind     1021 text -         -       error    -
root     rpcbind     1021    0 /dev         30 crw-rw-rw-    null rw
root     rpcbind     1021    6 /dev         30 crw-rw-rw-    null rw
root     watchdogd   1007 root -         -       error    -
root     watchdogd   1007   wd -         -       error    -
root     watchdogd   1007 text -         -       error    -
root     watchdogd   1007    0 /dev         30 crw-rw-rw-    null rw
root     syslogd     1004 root -         -       error    -
root     syslogd     1004   wd -         -       error    -
root     syslogd     1004 text -         -       error    -
root     syslogd     1004    0 /dev         30 crw-rw-rw-    null rw
root     syslogd     1004    6 /dev         30 crw-rw-rw-    null rw
root     syslogd     1004   12 /dev         30 crw-rw-rw-    null rw
root     syslogd     1004   18 -         -       error    -
root     devd         866 root -         -       error    -
root     devd         866   wd -         -       error    -
root     devd         866 text -         -       error    -
root     devd         866    0 /dev         30 crw-rw-rw-    null rw
root     devd         866    6 /dev         30 crw-rw-rw-    null rw
root     moused       847 root -         -       error    -
root     moused       847   wd -         -       error    -
root     moused       847 text -         -       error    -
root     moused       847    0 /dev         30 crw-rw-rw-    null rw
root     init           1 root -         -       error    -
root     init           1   wd -         -       error    -
root     init           1 text -         -       error    -
root     kernel         0 root -         -       error    -
root     kernel         0   wd -         -       error    -

------------------------------------------------------------------------
dmesg

Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
 	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.0-CURRENT #16: Sat Aug 10 13:45:06 CDT 2013
     root_at_borg.lerctr.org:/usr/obj/usr/src/sys/BORG-DTRACE amd64
FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610
CPU: Intel(R) Xeon(R) CPU           E5410  _at_ 2.33GHz (2327.55-MHz K8-class CPU)
   Origin = "GenuineIntel"  Id = 0x10676  Family = 0x6  Model = 0x17  Stepping = 6
   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
   Features2=0xce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1>
   AMD Features=0x20100800<SYSCALL,NX,LM>
   AMD Features2=0x1<LAHF>
   TSC: P-state invariant, performance statistics
real memory  = 68719476736 (65536 MB)
avail memory = 65660014592 (62618 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <PTLTD  	 APIC  >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s)
  cpu0 (BSP): APIC ID:  0
  cpu1 (AP): APIC ID:  1
  cpu2 (AP): APIC ID:  2
  cpu3 (AP): APIC ID:  3
  cpu4 (AP): APIC ID:  4
  cpu5 (AP): APIC ID:  5
  cpu6 (AP): APIC ID:  6
  cpu7 (AP): APIC ID:  7
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
kbd1 at kbdmux0
netmap: loaded module
random: <Software, Yarrow> initialized
cryptosoft0: <software crypto> on motherboard
acpi0: <PTLTD   RSDT> on motherboard
acpi0: Power Button (fixed)
unknown: I/O range not supported
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 350
Event timer "HPET1" frequency 14318180 Hz quality 340
Event timer "HPET2" frequency 14318180 Hz quality 340
atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci1
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci2
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 0.2 on pci3
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> irq 18 at device 2.0 on pci2
pci6: <ACPI PCI bus> on pcib6
em0: <Intel(R) PRO/1000 Network Connection 7.3.8> port 0x2000-0x201f mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0.0 on pci6
em0: Using an MSI interrupt
em0: Ethernet address: 00:30:48:f2:29:9c
001.000009 netmap_attach [2244] success for em0
em1: <Intel(R) PRO/1000 Network Connection 7.3.8> port 0x2020-0x203f mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0.1 on pci6
em1: Using an MSI interrupt
em1: Ethernet address: 00:30:48:f2:29:9d
001.000010 netmap_attach [2244] success for em1
pcib7: <ACPI PCI-PCI bridge> at device 0.3 on pci1
pci7: <ACPI PCI bus> on pcib7
pcib8: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci8: <ACPI PCI bus> on pcib8
vgapci0: <VGA-compatible display> port 0x3000-0x307f mem 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff irq 16 at device 0.0 on pci8
nvidia0: <GeForce GT 610> on vgapci0
vgapci0: child nvidia0 requested pci_enable_io
vgapci0: child nvidia0 requested pci_enable_io
pci8: <multimedia, HDA> at device 0.1 (no driver attached)
pcib9: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci9: <ACPI PCI bus> on pcib9
pci0: <base peripheral> at device 8.0 (no driver attached)
pcib10: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
pci10: <ACPI PCI bus> on pcib10
pcib11: <PCI-PCI bridge> irq 16 at device 0.0 on pci10
pci11: <PCI bus> on pcib11
pcm0: <Envy24HT audio (Generic)> port 0x4080-0x409f,0x4000-0x407f irq 16 at device 0.0 on pci11
pcm0: system configuration
   SubVendorID: 0x1412, SubDeviceID: 0x2403
   XIN2 Clock Source: 24.576MHz(96kHz*256)
   MPU-401 UART(s) #: not implemented
   ADC #: 1 and SPDIF receiver connected
   DAC #: 4
   Multi-track converter type: AC'97(SDATA_OUT:packed)
   S/PDIF(IN/OUT): 1/1 ID# 0x00
   GPIO(mask/dir/state): 0xff/0xff/0xff
uhci0: <Intel 631XESB/632XESB/3100 USB controller USB-1> port 0x1800-0x181f irq 17 at device 29.0 on pci0
usbus0 on uhci0
uhci1: <Intel 631XESB/632XESB/3100 USB controller USB-2> port 0x1820-0x183f irq 19 at device 29.1 on pci0
usbus1 on uhci1
uhci2: <Intel 631XESB/632XESB/3100 USB controller USB-3> port 0x1840-0x185f irq 18 at device 29.2 on pci0
usbus2 on uhci2
ehci0: <Intel 63XXESB USB 2.0 controller> mem 0xd9600400-0xd96007ff irq 17 at device 29.7 on pci0
usbus3: EHCI version 1.0
usbus3 on ehci0
pcib12: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci12: <ACPI PCI bus> on pcib12
vgapci1: <VGA-compatible display> port 0x5000-0x50ff mem 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 on pci12
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel 63XXESB2 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ahci0: <Intel ESB2 AHCI SATA controller> port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ichsmb0: <Intel 631xESB/6321ESB (ESB2) SMBus controller> port 0x1100-0x111f irq 19 at device 31.3 on pci0
smbus0: <System Management Bus> on ichsmb0
acpi_button0: <Power Button> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
ppc0: <Parallel port> port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/9 bytes threshold
ppbus0: <Parallel port bus> on ppc0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
ichwd0 on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
coretemp4: <CPU On-Die Thermal Sensors> on cpu4
est4: <Enhanced SpeedStep Frequency Control> on cpu4
p4tcc4: <CPU Frequency Thermal Control> on cpu4
coretemp5: <CPU On-Die Thermal Sensors> on cpu5
est5: <Enhanced SpeedStep Frequency Control> on cpu5
p4tcc5: <CPU Frequency Thermal Control> on cpu5
coretemp6: <CPU On-Die Thermal Sensors> on cpu6
est6: <Enhanced SpeedStep Frequency Control> on cpu6
p4tcc6: <CPU Frequency Thermal Control> on cpu6
coretemp7: <CPU On-Die Thermal Sensors> on cpu7
est7: <Enhanced SpeedStep Frequency Control> on cpu7
p4tcc7: <CPU Frequency Thermal Control> on cpu7
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
vboxdrv: fAsync=0 offMin=0x395 offMax=0x50f
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
ata0: DMA limited to UDMA33, controller found non-ATA66 cable
ada0 at ahcich0 bus 0 scbus1 target 0 lun 0
ada0: <ST2000DL003-9VT166 CC3D> ATA-8 SATA 3.x device
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada0: quirks=0x1<4K>
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus2 target 0 lun 0
ada1: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1: quirks=0x1<4K>
ada1: Previously was known as ad6
ada2 at ahcich2 bus 0 scbus3 target 0 lun 0
ada2: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada2: quirks=0x1<4K>
ada2: Previously was known as ad8
ada3 at ahcich3 bus 0 scbus4 target 0 lun 0
ada3: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada3: quirks=0x1<4K>
ada3: Previously was known as ad10
ada4 at ahcich4 bus 0 scbus5 target 0 lun 0
ada4: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada4: quirks=0x1<4K>
ada4: Previously was known as ad12
ada5 at ahcich5 bus 0 scbus6 target 0 lun 0
ada5: <ST2000DL003-9VT166 CC3D> ATA-8 SATA 3.x device
ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada5: quirks=0x1<4K>
ada5: Previously was known as ad14
cd0 at ata0 bus 0 scbus0 target 0 lun 0
cd0: <Memorex DVD+-RAM 510L v1 MWS7> Removable CD-ROM SCSI-0 device 
cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
SMP: AP CPU #6 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #5 Launched!
uhub1: 2 ports with 2 removable, self powered
uhub0: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
Root mount waiting for: usbus3
uhub3: 6 ports with 6 removable, self powered
Root mount waiting for: usbus3
Root mount waiting for: usbus3
ugen3.2: <Peppercon AG> at usbus3
ukbd0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus3
kbd2 at ukbd0
Trying to mount root from zfs:zroot/ROOT/default []...
ugen0.2: <Microsoft> at usbus0
ugen0.3: <American Power Conversion> at usbus0
ffclock reset: HPET (14318180 Hz), time = 1376160821.500000000
Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3.
Setting hostid: 0xf53a926e.
Entropy harvesting: interrupts ethernet point_to_point kickstart.
Starting file system checks:
Mounting local file systems:.
Writing entropy file:.
Setting hostname: borg.lerctr.org.
Starting Network: lo0 em0 em1.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
 	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
 	inet6 ::1 prefixlen 128
 	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
 	inet 127.0.0.1 netmask 0xff000000
 	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
 	options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
 	ether 00:30:48:f2:29:9c
 	inet 192.168.200.4 netmask 0xffffff00 broadcast 192.168.200.255
 	inet6 fe80::230:48ff:fef2:299c%em0 prefixlen 64 scopeid 0x1
 	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
 	media: Ethernet autoselect
 	status: no carrier
em1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
 	options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
 	ether 00:30:48:f2:29:9d
 	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
 	media: Ethernet autoselect
 	status: no carrier
Starting devd.
Starting Network: em1.
em1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
 	options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
 	ether 00:30:48:f2:29:9d
 	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
 	media: Ethernet autoselect
 	status: no carrier
uhid0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus3
ums0: <Microsoft Microsoft Trackball Optical, class 0/0, rev 1.10/1.21, addr 2> on usbus0
ums0: 5 buttons and [XYZ] coordinates ID=0
Starting ums0 moused.
add net default: gateway 192.168.200.11
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Mounting NFS file systems:.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/kde4/lib /usr/local/lib/compat /usr/local/lib/dbmail /usr/local/lib/event2 /usr/local/lib/pth /usr/local/lib/qt4 /usr/local/lib/virtualbox
32-bit compatibility ldconfig path: /usr/lib32 /usr/local/lib32/compat
Creating and/or trimming log files.
Starting syslogd.
Starting watchdogd.
No core dumps found.
Additional ABI support: linux.
Starting rpcbind.
NFS access cache time=60
Clearing /tmp (X related).
Starting mountd.
NFSv4 is disabled
Starting nfsd.
Starting apcupsd.
Updating motd:.
Mounting late file systems:.
Starting ntpd.
Starting sshblock.
Starting smartd.
Starting exim.
Starting cupsd.
Starting boinc_client.
Starting bacula_sd.
Starting bacula_fd.
Starting bacula_dir.
Performing sanity check on sshd configuration.
Starting sshd.
Configuring syscons: blanktime.
Starting cron.
mixer: WRITE_MIXER: Device not configured
mixer: WRITE_MIXER: Device not configured
mixer: WRITE_MIXER: Device not configured
mixer: WRITE_MIXER: Device not configured
mixer: WRITE_MIXER: Device not configured
mixer: WRITE_MIXER: Device not configured
Starting inetd.
Starting background file system checks in 60 seconds.

Sat Aug 10 13:54:18 CDT 2013


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0x5e
fault code		= supervisor read data, page not present
instruction pointer	= 0x20:0xffffffff80709be9
stack pointer	        = 0x28:0xffffff900d55c860
frame pointer	        = 0x28:0xffffff900d55c870
code segment		= base 0x0, limit 0xfffff, type 0x1b
 			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 1378 (cpucontrol)
trap number		= 12
panic: page fault
cpuid = 0
Uptime: 1m48s
Dumping 2337 out of 64479 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%

------------------------------------------------------------------------
kernel config

options	CONFIG_AUTOGENERATED
ident	BORG-DTRACE
machine	amd64
cpu	HAMMER
makeoptions	WITH_CTF=1
makeoptions	DEBUG=-g
options	FFCLOCK
options	USB_DEBUG
options	RDRAND_RNG
options	PADLOCK_RNG
options	ATH_ENABLE_11N
options	AH_AR5416_INTERRUPT_MITIGATION
options	AH_SUPPORT_AR5416
options	IEEE80211_SUPPORT_MESH
options	IEEE80211_AMPDU_AGE
options	IEEE80211_DEBUG
options	SC_PIXEL_MODE
options	VESA
options	AHD_REG_PRETTY_PRINT
options	AHC_REG_PRETTY_PRINT
options	ATA_STATIC_ID
options	SMP
options	MALLOC_DEBUG_MAXZONES=8
options	KDB_TRACE
options	INCLUDE_CONFIG_FILE
options	DDB_CTF
options	KDTRACE_HOOKS
options	KDTRACE_FRAME
options	MAC
options	CAPABILITIES
options	CAPABILITY_MODE
options	AUDIT
options	HWPMC_HOOKS
options	KBD_INSTALL_CDEV
options	PRINTF_BUFR_SIZE=128
options	_KPOSIX_PRIORITY_SCHEDULING
options	SYSVSEM
options	SYSVMSG
options	SYSVSHM
options	STACK
options	KTRACE
options	SCSI_DELAY=5000
options	COMPAT_FREEBSD7
options	COMPAT_FREEBSD6
options	COMPAT_FREEBSD5
options	COMPAT_FREEBSD4
options	COMPAT_FREEBSD32
options	GEOM_LABEL
options	GEOM_RAID
options	GEOM_PART_GPT
options	PSEUDOFS
options	PROCFS
options	CD9660
options	MSDOSFS
options	NFS_ROOT
options	NFSLOCKD
options	NFSD
options	NFSCL
options	QUOTA
options	SCTP
options	TCP_OFFLOAD
options	INET6
options	INET
options	PREEMPTION
options	SCHED_ULE
options	NEW_PCIB
options	GEOM_PART_MBR
options	GEOM_PART_EBR_COMPAT
options	GEOM_PART_EBR
options	GEOM_PART_BSD
device	isa
device	mem
device	io
device	uart_ns8250
device	cpufreq
device	acpi
device	pci
device	fdc
device	ahci
device	ata
device	esp
device	isci
device	scbus
device	ch
device	da
device	sa
device	cd
device	pass
device	ses
device	hptnr
device	aacraid
device	atkbdc
device	atkbd
device	psm
device	kbdmux
device	vga
device	splash
device	sc
device	agp
device	uart
device	ppc
device	ppbus
device	lpt
device	ppi
device	em
device	miibus
device	cas
device	gem
device	hme
device	nfe
device	nge
device	loop
device	random
device	ether
device	vlan
device	tun
device	md
device	gif
device	faith
device	firmware
device	bpf
device	uhci
device	ohci
device	ehci
device	xhci
device	usb
device	ukbd
device	umass
device	virtio
device	virtio_pci
device	vtnet
device	virtio_blk
device	virtio_scsi
device	virtio_balloon
device	netmap

------------------------------------------------------------------------
ddb capture buffer

ddb: ddb_capture: kvm_nlist

-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640                 E-Mail: ler_at_lerctr.org
US Mail: 108 Turvey Cove, Hutto, TX 78634-5688
Received on Sat Aug 10 2013 - 17:06:24 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:40 UTC