Re: Panic _at_r207844; current process: flowcleaner "Fatal trap 12: page fault while in kernel mode"

From: David Wolfskill <david_at_catwhisker.org>
Date: Mon, 10 May 2010 15:52:07 -0700
On Mon, May 10, 2010 at 03:39:11PM -0700, K. Macy wrote:
> Are you not able to dump core?
> ....

Here's the crash summary; I can put the dump on my Web server on request.
(It weighs in at 89MB.)

freebeast.catwhisker.org dumped core - see /var/crash/vmcore.0

Mon May 10 15:47:10 PDT 2010

FreeBSD freebeast.catwhisker.org 9.0-CURRENT FreeBSD 9.0-CURRENT #155 r207844M: Mon May 10 14:55:41 PDT 2010     root_at_freebeast.catwhisker.org:/usr/obj/usr/src/sys/GENERIC  i386

panic: from debugger

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:
<118>Starting ypbind.
<118>Starting amd.
Kernel page fault with the following non-sleepable locks held:
exclusive sleep mutex mld_mtx (mld_mtx) r = 0 (0xc0f96468) locked _at_ /usr/src/sys/netinet6/mld6.c:1352
exclusive sleep mutex in6_multi_mtx (in6_multi_mtx) r = 0 (0xc0f95408) locked _at_ /usr/src/sys/netinet6/mld6.c:1351
KDB: stack backtrace:
db_trace_self_wrapper(c0cc72a5,c5213864,c08e9f65,c0ce3e07,547,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0ce3e07,547,ffffffff,c0f65a1c,c521389c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc982b,c52138b0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cffec8,c55810a4,c557f7f8,...) at witness_warn+0x1fd
trap(c521393c) at trap+0x19e
calltrap() at calltrap+0x6
--- trap 0xc, eip = 0xc0a4d340, esp = 0xc521397c, ebp = 0xc5213b14 ---
ip6_output(c5a4fc00,c0f964a0,0,1,c5213b44,...) at ip6_output+0x840
mld_dispatch_packet(0,0,c5213c1c,c0a509a7,c56fca2c,...) at mld_dispatch_packet+0x30d
mld_dispatch_queue(c56fca2c,0,c0ce3e07,591,c0cbb984,...) at mld_dispatch_queue+0x38
mld_fasttimo(c5213c48,c0904cc9,c5213c48,c0893aa4,c0e24f80,...) at mld_fasttimo+0x747
icmp6_fasttimo(c5213c48,c0893aa4,c0e24f80,4,c0cc24d6,...) at icmp6_fasttimo+0x8
pffasttimo(0,0,c0cc5113,189,c0e24fb8,...) at pffasttimo+0x29
softclock(c0e24f80,c5213cc8,c0893aa4,c0e28940,c5588838,...) at softclock+0x24a
intr_event_execute_handlers(c557f7f8,c5588800,c0cbf33f,533,c5588870,...) at intr_event_execute_handlers+0x125
ithread_loop(c557e120,c5213d38,c0cbf0b8,343,c557f7f8,...) at ithread_loop+0x9f
fork_exit(c087c780,c557e120,c5213d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xc5213d70, ebp = 0 ---


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0x3c
fault code		= supervisor read, page not present
instruction pointer	= 0x20:0xc0a4d340
stack pointer	        = 0x28:0xc521397c
frame pointer	        = 0x28:0xc5213b14
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, def32 1, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 12 (swi4: clock)
panic: from debugger
cpuid = 0
Uptime: 20s
Physical memory: 2030 MB
Dumping 89 MB: 74 58 42 26 10

#0  doadump () at pcpu.h:246
246	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) #0  doadump () at pcpu.h:246
#1  0xc08a384e in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:416
#2  0xc08a3b22 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:590
#3  0xc04d2c97 in db_panic (addr=Could not find the frame base for "db_panic".
) at /usr/src/sys/ddb/db_command.c:478
#4  0xc04d32c1 in db_command (last_cmdp=0xc0df1d5c, cmd_table=0x0, dopager=1)
    at /usr/src/sys/ddb/db_command.c:445
#5  0xc04d341a in db_command_loop () at /usr/src/sys/ddb/db_command.c:498
#6  0xc04d52bd in db_trap (type=12, code=0) at /usr/src/sys/ddb/db_main.c:229
#7  0xc08d5ee6 in kdb_trap (type=12, code=0, tf=0xc521393c)
    at /usr/src/sys/kern/subr_kdb.c:535
#8  0xc0be703f in trap_fatal (frame=0xc521393c, eva=60)
    at /usr/src/sys/i386/i386/trap.c:931
#9  0xc0be78ac in trap (frame=0xc521393c) at /usr/src/sys/i386/i386/trap.c:328
#10 0xc0bc8ecb in calltrap () at /usr/src/sys/i386/i386/exception.s:165
#11 0xc0a4d340 in ip6_output (m0=0xc5a4fc00, opt=0xc0f964a0, ro=0xc5213a68, 
    flags=1, im6o=0xc5213b44, ifpp=0xc5213b64, inp=0x0)
    at /usr/src/sys/netinet6/ip6_output.c:600
#12 0xc0a4ec3d in mld_dispatch_packet (m=0xc5a4fd00)
    at /usr/src/sys/netinet6/mld6.c:3119
#13 0xc0a4efe8 in mld_dispatch_queue (ifq=0xc5213be8, limit=0)
    at /usr/src/sys/netinet6/mld6.c:417
#14 0xc0a509a7 in mld_fasttimo () at /usr/src/sys/netinet6/mld6.c:1431
#15 0xc0a34158 in icmp6_fasttimo () at /usr/src/sys/netinet6/icmp6.c:2240
#16 0xc0904cc9 in pffasttimo (arg=0x0) at /usr/src/sys/kern/uipc_domain.c:531
#17 0xc08b6bda in softclock (arg=0xc0e24f80)
    at /usr/src/sys/kern/kern_timeout.c:430
#18 0xc087ba45 in intr_event_execute_handlers (p=0xc557f7f8, ie=0xc5588800)
    at /usr/src/sys/kern/kern_intr.c:1220
#19 0xc087c81f in ithread_loop (arg=0xc557e120)
    at /usr/src/sys/kern/kern_intr.c:1233
#20 0xc0879798 in fork_exit (callout=0xc087c780 <ithread_loop>, 
    arg=0xc557e120, frame=0xc5213d38) at /usr/src/sys/kern/kern_fork.c:843
#21 0xc0bc8f40 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:270
(kgdb) 

------------------------------------------------------------------------
ps -axl

  UID   PID  PPID CPU PRI NI    VSZ    RSS MWCHAN STAT  TT       TIME COMMAND
    0     0     0   0 -16  0      0      0 deadlk DLs   ??  6689708:24.00 [kernel]
    0     1     0   0  76  0   8032      0 wait   DLs   ??  7839166:30.00 [init]
    0     2     0   0  -8  0      0      0 -      DL    ??  1613170:30.00 [g_event]
    0     3     0   0  -8  0      0      0 -      DL    ??  806945:51.00 [g_up]
    0     4     0   0  -8  0      0      0 -      DL    ??  890180:42.00 [g_down]
    0     5     0   0 -16  0      0      0 aifthd DL    ??    0:00.00 [aac0aif]
    0     6     0   0 -16  0      0      0 -      DL    ??  4397:15.00 [fdc0]
    0     7     0   0  76  0      0      0 ccb_sc DL    ??    0:00.00 [xpt_thrd
    0     8     0   0  76  0      0      0 psleep DL    ??  8770:39.00 [pagedaem
    0     9     0   0  76  0      0      0 psleep DL    ??  238:03.00 [vmdaemon
    0    10     0   0 -16  0      0      0 audit_ DL    ??    0:00.00 [audit]
    0    11     0   0 171  0      0      0 -      RL    ??  -17871428:-11.55 [idle]
    0    12     0   0 -48  0      0      0 -      WL    ??  5797222:57.00 [intr]
    0    13     0   0 -16  0      0      0 -      DL    ??  138261:00.00 [yarrow]
    0    14     0   0 -64  0      0      0 -      DL    ??  13587:54.00 [usb]
    0    15     0   0  76  0      0      0 pgzero DL    ??  357:36.00 [pagezero
    0    16     0   0  76  0      0      0 psleep DL    ??  5494:21.00 [bufdaemo
    0    17     0   0  76  0      0      0 syncer DL    ??  12605:51.00 [syncer]
    0    18     0   0  76  0      0      0 vlruwt DL    ??  4270:03.00 [vnlru]
    0    19     0   0  76  0      0      0 sdflus DL    ??  104670:27.00 [softdepf
    0    20     0   0  76  0      0      0 flowcl DL    ??  3432:27.00 [flowclea
    0    21     1   0  76  0   9800      0 wait   Ds+   ??  3872524:03.00 [sh]
    0   660     1   0  76  0   8032      0 select Ds    ??  57910:21.00 [devd]
    0   857     1   0  76  0   9528      0 select Ds    ??  767461:39.00 [syslogd]
    0   881     1   0  76  0   9688      0 select Ds    ??  466142:42.00 [rpcbind]
    0   908     1   0  76  0   9460      0 select Ds    ??  295696:21.00 [ypbind]
    0   915    21   0  76  0   9800      0 wait   D+    ??  313004:15.00 [sh]
    0   921   915   0  76  0    328      0 vnread D+    ??    0:00.00 [sh]

------------------------------------------------------------------------
vmstat -s

    41366 cpu context switches
     4837 device interrupts
     3046 software interrupts
    70181 traps
   104641 system calls
       20 kernel threads created
      879  fork() calls
       22 vfork() calls
        0 rfork() calls
        0 swap pager pageins
        0 swap pager pages paged in
        0 swap pager pageouts
        0 swap pager pages paged out
      145 vnode pager pageins
      843 vnode pager pages paged in
        0 vnode pager pageouts
        0 vnode pager pages paged out
        0 page daemon wakeups
        0 pages examined by the page daemon
       28 pages reactivated
    24192 copy-on-write faults
       19 copy-on-write optimized faults
    28560 zero fill pages zeroed
     1048 zero fill pages prezeroed
        2 intransit blocking page faults
    69993 total VM faults taken
        0 pages affected by kernel thread creation
   839931 pages affected by  fork()
    21560 pages affected by vfork()
        0 pages affected by rfork()
        0 pages cached
    64820 pages freed
        0 pages freed by daemon
        0 pages freed by exiting processes
      560 pages active
      777 pages inactive
        0 pages in VM cache
     8298 pages wired down
   500911 pages free
     4096 bytes per page
     9658 total name lookups
          cache hits (84% pos + 8% neg) system 0% per-directory
          deletions 0%, falsehits 0%, toolong 0%

------------------------------------------------------------------------
vmstat -m

         Type InUse MemUse HighUse Requests  Size(s)
         cdev    10     2K       -       10  128
     pci_link    16     2K       -       16  32,64,128
        sigio     1     1K       -        1  32
     filedesc    28     7K       -      922  256
         kenv    79     7K       -       83  16,32,64,128,4096
    proc-args     6     1K       -      259  32,64,128
      ithread   125    12K       -      125  16,64,128
     SCSI SES     1     2K       -        1  2048
       KTRACE   100    13K       -      100  128
       linker   113     4K       -      115  16,32,512
        lockf    17     1K       -       17  32,64
       ip6ndp     5     1K       -        6  64,128
         temp    22   229K       -     5555  16,32,64,128,256,512,1024,2048,4096
       devbuf  3945  3999K       -     3991  16,32,64,128,256,512,1024,2048,4096
       module   457    29K       -      457  64,128
      entropy  1024    64K       -     1024  64
     mtx_pool     2     8K       -        2  4096
      subproc   101   131K       -      995  256,4096
         proc     2     8K       -        2  4096
      session     6     1K       -        6  64
         pgrp     6     1K       -        6  64
         cred    28     3K       -     1832  64,128
      uidinfo     2     2K       -        2  64,1024
       plimit     2     1K       -       36  256
    sysctltmp     0     0K       -      212  16,32,64,128,4096
    sysctloid  3114    95K       -     3250  16,32,64,128
       sysctl     0     0K       -      456  16,32,64
      callout     1   256K       -        1  
         umtx   264    25K       -      264  64,128
     p1003.1b     1     1K       -        1  16
         SWAP     4  4377K       -        4  64
       bus-sc    63   194K       -     2974  16,32,64,128,256,512,1024,2048,4096
          bus  1112    51K       -     5655  16,32,64,128,256,512,1024
      devstat    14    29K       -       14  16,4096
 eventhandler    81     4K       -       81  32,64,128
         kobj   330   660K       -      391  2048
      Per-cpu     1     1K       -        1  16
         UART     6     3K       -        6  16,256,1024
  ddb_capture     1    48K       -        1  
         rman   195    12K       -      693  16,32,64
       aacbuf   261    41K       -      261  32,128
         sbuf     0     0K       -     1198  16,32,64,128,256,512,1024,2048,4096
        stack     0     0K       -        2  128
    taskqueue    15     1K       -       15  16,64
       Unitno    11     1K       -       55  16,64
       acpica  1370    68K       -    66145  16,32,64,128,256,512,1024
     acpitask     1     1K       -        1  1024
      Witness     1   104K       -        1  
          iov     0     0K       -      305  16,64,128,256
       select     6     1K       -        6  64
     ioctlops     0     0K       -      520  16,32,64,128,512,1024
          msg     4    25K       -        4  1024,4096
          sem     4     6K       -        4  256,512,1024,4096
          shm     1    12K       -        1  
          tty    21    11K       -       23  512,2048
     mbuf_tag     0     0K       -       13  32,64
         ksem     1     4K       -        1  4096
        shmfd     1     4K       -        1  4096
          pcb    19    79K       -       26  16,64,512,1024,2048,4096
       soname     4     1K       -      123  16,32,128
     vfscache     1   512K       -        1  
     vfs_hash     1   256K       -        1  
       vnodes     2     1K       -        2  128
       USBdev    20     6K       -       20  32,128,1024
  vnodemarker     0     0K       -       33  512
        mount   188     7K       -      232  16,32,64,128,256
          BPF     3     1K       -        3  64
  ether_multi    40     2K       -       46  16,32,64
       ifaddr    43     9K       -       43  16,32,64,128,256,512,2048
        ifnet     4     4K       -        4  64,1024
          USB    35     6K       -       35  16,32,64,1024
        clone     6    24K       -        6  4096
       arpcom     2     1K       -        2  16
      lltable    15     4K       -       15  128,256
      CAM XPT    44    14K       -      415  16,32,64,256,1024,2048
      acpisem    18     3K       -       18  64,128
CAM dev queue     3     1K       -        3  128
     routetbl   101  8212K       -      178  16,32,64,128,256
         igmp     3     1K       -        3  128
    CAM queue    19     1K       -      589  16,32
       kbdmux     6    18K       -        6  16,256,1024,2048
          LED     2     1K       -        2  64
     in_multi     2     1K       -        2  128
    sctp_iter     0     0K       -        3  128
     sctp_ifn     2     1K       -        2  128
     sctp_ifa     5     1K       -        5  128
     sctp_vrf     1     1K       -        1  64
    sctp_a_it     0     0K       -        3  16
    hostcache     1    16K       -        1  
     syncache     1    72K       -        1  
       DEVFS1   117    30K       -      139  256
 ip6_moptions     2     1K       -        2  32,128
    in6_multi    22     3K       -       22  16,256
  in6_mfilter     1     1K       -        1  512
       DEVFS3   134    17K       -      157  128
          mld     3     1K       -        3  128
      NFS FHA     1     1K       -        1  1024
          rpc     2     5K       -        2  128,4096
audit_evclass   172     3K       -      211  16
        sbdep     0     0K       -        1  32
     freework     0     0K       -        7  64
       dirrem     1     1K       -       17  64
       diradd    13     1K       -       18  64
     freefile     0     0K       -       14  32
     freeblks     0     0K       -        7  64
     freefrag     0     0K       -        1  64
       newblk     6    65K       -       14  128
    bmsafemap     5     5K       -        7  128,4096
     inodedep    17   260K       -       35  256
      pagedep     5    65K       -        6  128
  ufs_dirhash    27     5K       -       27  16,32,64,512
    ufs_mount    33    77K       -       33  256,2048,4096
        DEVFS    17     1K       -       18  16,64
    vm_pgdata     3    65K       -        3  64
     atkbddev     2     1K       -        2  32
    pfs_nodes    21     3K       -       21  128
      CAM SIM     3     1K       -        3  128
       apmdev     1     1K       -        1  64
         GEOM   193    58K       -     1321  16,32,64,128,512,1024,2048
  ata_generic     1     1K       -        1  1024
      io_apic     3     3K       -        3  1024
      memdesc     1     4K       -        1  4096
     nexusdev     4     1K       -        4  16
   acd_driver     1     2K       -        1  2048
       isadev     5     1K       -        5  64
   CAM periph     8     1K       -       83  16,32,64,128
      acpidev    62     2K       -       62  32

------------------------------------------------------------------------
vmstat -z

ITEM                     SIZE     LIMIT      USED      FREE  REQUESTS  FAILURES

UMA Kegs:                 128,        0,       88,        2,       88,        0
UMA Zones:                888,        0,       88,        0,       88,        0
UMA Slabs:                284,        0,      844,       10,     1790,        0
UMA RCntSlabs:            544,        0,      195,        1,      195,        0
UMA Hash:                 128,        0,        4,       26,        4,        0
16 Bucket:                 76,        0,       42,        8,       63,        0
32 Bucket:                140,        0,       35,       21,       56,        0
64 Bucket:                268,        0,       34,        8,       80,       13
128 Bucket:               524,        0,       44,        5,     1078,      112
VM OBJECT:                136,        0,      495,      114,    10183,        0
MAP:                      140,        0,        7,       21,        7,        0
KMAP ENTRY:                72,    57505,       34,      231,     4441,        0
MAP ENTRY:                 72,        0,       82,      236,    18673,        0
DP fakepg:                 72,        0,        0,        0,        0,        0
SG fakepg:                 72,        0,        0,        0,        0,        0
mt_zone:                 2056,        0,      282,        0,      282,        0
16:                        16,        0,     2744,      504,    36762,        0
32:                        32,        0,     2696,      242,    38485,        0
64:                        64,        0,     4416,      304,     6716,        0
128:                      128,        0,     2538,      222,     6347,        0
256:                      256,        0,      584,       61,     2556,        0
512:                      512,        0,       55,       41,     1903,        0
1024:                    1024,        0,       42,      154,     1121,        0
2048:                    2048,        0,      359,       41,      642,        0
4096:                    4096,        0,      397,       53,     6095,        0
Files:                     56,        0,       42,      226,     3377,        0
TURNSTILE:                 72,        0,      133,       47,      133,        0
umtx pi:                   52,        0,        0,        0,        0,        0
MAC labels:                20,        0,        0,        0,        0,        0
PROC:                     680,        0,       27,       45,      921,        0
THREAD:                   624,        0,      112,       20,      112,        0
SLEEPQUEUE:                48,        0,      133,      103,      133,        0
VMSPACE:                  240,        0,        8,       72,      903,        0
cpuset:                    40,        0,        2,      182,        2,        0
audit_record:             816,        0,        0,        0,        0,        0
mbuf_packet:              256,        0,      256,      133,      537,        0
mbuf:                     256,        0,        6,      255,       74,        0
mbuf_cluster:            2048,    25600,      384,        6,      384,        0
mbuf_jumbo_page:         4096,    12800,        0,        0,        0,        0
mbuf_jumbo_9k:           9216,    19200,        0,        0,        0,        0
mbuf_jumbo_16k:         16384,    12800,        0,        0,        0,        0
mbuf_ext_refcnt:            4,        0,        0,        0,        0,        0
ttyinq:                   152,        0,       15,       37,       15,        0
ttyoutq:                  256,        0,        8,       22,        8,        0
g_bio:                    140,        0,        4,      108,     4426,        0
ata_request:              208,        0,        0,       38,       18,        0
ata_composite:            180,        0,        0,        0,        0,        0
VNODE:                    272,        0,      383,       23,      398,        0
VNODEPOLL:                 60,        0,        0,        0,        0,        0
S VFS Cache:               72,        0,      350,       74,      628,        0
L VFS Cache:              292,        0,        0,        0,        0,        0
NAMEI:                   1024,        0,        1,       35,     4632,        0
NFSMOUNT:                 524,        0,        0,        0,        0,        0
NFSNODE:                  468,        0,        0,        0,        0,        0
DIRHASH:                 1024,        0,       25,       11,       25,        0
pipe:                     392,        0,        0,       40,      621,        0
ksiginfo:                  80,        0,       54,     1002,       55,        0
itimer:                   220,        0,        0,        0,        0,        0
KNOTE:                     72,        0,        0,        0,        0,        0
socket:                   412,    25605,       15,       21,      200,        0
unpcb:                    172,    25622,        4,       65,       21,        0
ipq:                       32,      904,        0,        0,        0,        0
udp_inpcb:                220,    25614,        8,       46,      171,        0
udpcb:                      8,    25781,        8,      398,      171,        0
tcp_inpcb:                220,    25614,        3,       51,        3,        0
tcpcb:                    632,    25602,        3,       15,        3,        0
tcptw:                     52,     5184,        0,        0,        0,        0
syncache:                 112,    15365,        0,        0,        0,        0
hostcache:                 76,    15400,        0,        0,        0,        0
tcpreass:                  20,     1690,        0,        0,        0,        0
sackhole:                  20,        0,        0,        0,        0,        0
sctp_ep:                  860,    25600,        0,        0,        0,        0
sctp_asoc:               1484,    40000,        0,        0,        0,        0
sctp_laddr:                24,    80040,        0,      290,        4,        0
sctp_raddr:               432,    80001,        0,        0,        0,        0
sctp_chunk:                92,   400008,        0,        0,        0,        0
sctp_readq:                76,   400000,        0,        0,        0,        0
sctp_stream_msg_out:       64,   400020,        0,        0,        0,        0
sctp_asconf:               24,   400055,        0,        0,        0,        0
sctp_asconf_ack:           24,   400055,        0,        0,        0,        0
ripcb:                    220,    25614,        0,        0,        0,        0
rtentry:                  108,        0,       17,       91,       17,        0
selfd:                     28,        0,       21,      360,      390,        0
ip4flow:                   40,    50232,        8,      268,        8,        0
ip6flow:                   64,    50228,        0,        0,        0,        0
Mountpoints:              648,        0,       12,       12,       12,        0
FFS inode:                116,        0,      350,       46,      364,        0
FFS1 dinode:              128,        0,        0,        0,        0,        0
FFS2 dinode:              256,        0,      350,       40,      364,        0
SWAPMETA:                 276,   121576,        0,        0,        0,        0


------------------------------------------------------------------------
vmstat -i

interrupt                          total       rate
irq1: atkbd0                           6          0
irq4: uart0                         2305         67
irq6: fdc0                             6          0
irq15: ata1                           35          1
irq16: uhci0 uhci3                  1235         36
irq24: aac0                         1221         35
irq54: em0                            29          0
cpu0: timer                        31220        918
cpu1: timer                        25893        761
Total                              61950       1822

------------------------------------------------------------------------
pstat -T

 42/12328 files
0M/20479M swap space

------------------------------------------------------------------------
pstat -s

Device          512-blocks     Used    Avail Capacity
/dev/aacd0s4b     20971264        0 20971264     0%
/dev/aacd1s1b     20971264        0 20971264     0%
Total             41942528        0 41942528     0%

------------------------------------------------------------------------
iostat

iostat: kvm_read(_tk_nin): invalid address (0x0)
iostat: disabling TTY statistics
iostat: kvm_getcptime: invalid address (0x0)
iostat: disabling CPU time statistics
           aacd0            aacd1            pass0 
  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s 
  8.70  29  0.25   4.49   4  0.02   0.00   0  0.00 

------------------------------------------------------------------------
ipcs -a

Message Queues:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME   

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME   

Semaphores:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME   


------------------------------------------------------------------------
ipcs -T

msginfo:
	msgmax:        16384	(max characters in a message)
	msgmni:           40	(# of message queues)
	msgmnb:         2048	(max characters in a message queue)
	msgtql:           40	(max # of messages in system)
	msgssz:            8	(size of a message segment)
	msgseg:         2048	(# of message segments in system)

shminfo:
	shmmax:     33554432	(max shared memory segment size)
	shmmin:            1	(min shared memory segment size)
	shmmni:          192	(max number of shared memory identifiers)
	shmseg:          128	(max shared memory segments per process)
	shmall:         8192	(max amount of shared memory in pages)

seminfo:
	semmap:           30	(# of entries in semaphore map)
	semmni:           10	(# of semaphore identifiers)
	semmns:           60	(# of semaphores in system)
	semmnu:           30	(# of undo structures in system)
	semmsl:           60	(max # of semaphores per id)
	semopm:          100	(max # of operations per semop call)
	semume:           10	(max # of undo entries per process)
	semusz:          136	(size in bytes of undo structure)
	semvmx:        32767	(semaphore maximum value)
	semaem:        16384	(adjust on exit max value)


------------------------------------------------------------------------
nfsstat

Client Info:
Rpc Counts:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Rpc Info:
 TimedOut   Invalid X Replies   Retries  Requests
        0         0         0         0         0
Cache Info:
Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW Hits    Misses
        0         0         0         0         0         0         0         0
BioRLHits    Misses BioD Hits    Misses DirE Hits    Misses
        0         0         0         0         0         0

Server Info:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Server Ret-Failed
                0
Server Faults
            0
Server Cache Stats:
   Inprog      Idem  Non-idem    Misses
        0         0         0         0
Server Write Gathering:
 WriteOps  WriteRPC   Opsaved
        0         0         0

------------------------------------------------------------------------
netstat -s

tcp:
	0 packets sent
		0 data packets (0 bytes)
		0 data packets (0 bytes) retransmitted
		0 data packets unnecessarily retransmitted
		0 resends initiated by MTU discovery
		0 ack-only packets (0 delayed)
		0 URG only packets
		0 window probe packets
		0 window update packets
		0 control packets
	0 packets received
		0 acks (for 0 bytes)
		0 duplicate acks
		0 acks for unsent data
		0 packets (0 bytes) received in-sequence
		0 completely duplicate packets (0 bytes)
		0 old duplicate packets
		0 packets with some dup. data (0 bytes duped)
		0 out-of-order packets (0 bytes)
		0 packets (0 bytes) of data after window
		0 window probes
		0 window update packets
		0 packets received after close
		0 discarded for bad checksums
		0 discarded for bad header offset fields
		0 discarded because packet too short
		0 discarded due to memory problems
	0 connection requests
	0 connection accepts
	0 bad connection attempts
	0 listen queue overflows
	0 ignored RSTs in the windows
	0 connections established (including accepts)
	0 connections closed (including 0 drops)
		0 connections updated cached RTT on close
		0 connections updated cached RTT variance on close
		0 connections updated cached ssthresh on close
	0 embryonic connections dropped
	0 segments updated rtt (of 0 attempts)
	0 retransmit timeouts
		0 connections dropped by rexmit timeout
	0 persist timeouts
		0 connections dropped by persist timeout
	0 Connections (fin_wait_2) dropped because of timeout
	0 keepalive timeouts
		0 keepalive probes sent
		0 connections dropped by keepalive
	0 correct ACK header predictions
	0 correct data packet header predictions
	0 syncache entries added
		0 retransmitted
		0 dupsyn
		0 dropped
		0 completed
		0 bucket overflow
		0 cache overflow
		0 reset
		0 stale
		0 aborted
		0 badack
		0 unreach
		0 zone failures
	0 cookies sent
	0 cookies received
	0 SACK recovery episodes
	0 segment rexmits in SACK recovery episodes
	0 byte rexmits in SACK recovery episodes
	0 SACK options (SACK blocks) received
	0 SACK options (SACK blocks) sent
	0 SACK scoreboard overflow
	0 packets with ECN CE bit set
	0 packets with ECN ECT(0) bit set
	0 packets with ECN ECT(1) bit set
	0 successful ECN handshakes
	0 times ECN reduced the congestion window
udp:
	12 datagrams received
	0 with incomplete header
	0 with bad data length field
	0 with bad checksum
	0 with no checksum
	3 dropped due to no socket
	0 broadcast/multicast datagrams undelivered
	0 dropped due to full socket buffers
	0 not for hashed pcb
	9 delivered
	7 datagrams output
	0 times multicast source filter matched
ip:
	8 total packets received
	0 bad header checksums
	0 with size smaller than minimum
	0 with data size < data length
	0 with ip length > max ip packet size
	0 with header length < data size
	0 with data length < header length
	0 with bad options
	0 with incorrect version number
	0 fragments received
	0 fragments dropped (dup or out of space)
	0 fragments dropped after timeout
	0 packets reassembled ok
	8 packets for this host
	0 packets for unknown/unsupported protocol
	0 packets forwarded (0 packets fast forwarded)
	0 packets not forwardable
	0 packets received for unknown multicast group
	0 redirects sent
	5 packets sent from this host
	0 packets sent with fabricated ip header
	0 output packets dropped due to no bufs, etc.
	0 output packets discarded due to no route
	0 output datagrams fragmented
	0 fragments created
	0 datagrams that can't be fragmented
	0 tunneling packets that can't find gif
	0 datagrams with bad address in header
icmp:
	3 calls to icmp_error
	0 errors not generated in response to an icmp message
	Output histogram:
		destination unreachable: 3
	0 messages with bad code fields
	0 messages less than the minimum length
	0 messages with bad checksum
	0 messages with bad length
	0 multicast echo requests ignored
	0 multicast timestamp requests ignored
	0 message responses generated
	0 invalid return addresses
	0 no return routes
igmp:
	0 messages received
	0 messages received with too few bytes
	0 messages received with wrong TTL
	0 messages received with bad checksum
	0 V1/V2 membership queries received
	0 V3 membership queries received
	0 membership queries received with invalid field(s)
	0 general queries received
	0 group queries received
	0 group-source queries received
	0 group-source queries dropped
	0 membership reports received
	0 membership reports received with invalid field(s)
	0 membership reports received for groups to which we belong
	0 V3 reports received without Router Alert
	0 membership reports sent
arp:
	3 ARP requests sent
	0 ARP replies sent
	12 ARP requests received
	2 ARP replies received
	14 ARP packets received
	0 total packets dropped due to no ARP entry
	0 ARP entrys timed out
	0 Duplicate IPs seen
ip6:
	4 total packets received
	0 with size smaller than minimum
	0 with data size < data length
	0 with bad options
	0 with incorrect version number
	0 fragments received
	0 fragments dropped (dup or out of space)
	0 fragments dropped after timeout
	0 fragments that exceeded limit
	0 packets reassembled ok
	4 packets for this host
	0 packets forwarded
	0 packets not forwardable
	0 redirects sent
	11 packets sent from this host
	0 packets sent with fabricated ip header
	0 output packets dropped due to no bufs, etc.
	0 output packets discarded due to no route
	0 output datagrams fragmented
	0 fragments created
	0 datagrams that can't be fragmented
	0 packets that violated scope rules
	0 multicast packets which we don't join
	Input histogram:
		UDP: 4
	Mbuf statistics:
		5 one mbuf
		0 one ext mbuf
		0 two or more ext mbuf
	0 packets whose headers are not continuous
	0 tunneling packets that can't find gif
	0 packets discarded because of too many headers
	0 failures of source address selection
	Source addresses selection rule applied:
		5 first candidate
		4 matching label
icmp6:
	0 calls to icmp6_error
	0 errors not generated in response to an icmp6 message
	0 errors not generated because of rate limitation
	Output histogram:
		neighbor solicitation: 1
	0 messages with bad code fields
	0 messages < minimum length
	0 bad checksums
	0 messages with bad length
	Histogram of error messages to be generated:
		0 no route
		0 administratively prohibited
		0 beyond scope
		0 address unreachable
		0 port unreachable
		0 packet too big
		0 time exceed transit
		0 time exceed reassembly
		0 erroneous header field
		0 unrecognized next header
		0 unrecognized option
		0 redirect
		0 unknown
	0 message responses generated
	0 messages with too many ND options
	0 messages with bad ND options
	0 bad neighbor solicitation messages
	0 bad neighbor advertisement messages
	0 bad router solicitation messages
	0 bad router advertisement messages
	0 bad redirect messages
	0 path MTU changes
rip6:
	0 messages received
	0 checksum calculations on inbound
	0 messages with bad checksum
	0 messages dropped due to no socket
	0 multicast messages dropped due to no socket
	0 messages dropped due to full socket buffers
	0 delivered
	0 datagrams output

------------------------------------------------------------------------
netstat -m

262/388/650 mbufs in use (current/cache/total)
251/139/390/25600 mbuf clusters in use (current/cache/total/max)
256/133 mbuf+clusters out of packet secondary zone in use (current/cache)
0/0/0/12800 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/19200 9k jumbo clusters in use (current/cache/total/max)
0/0/0/12800 16k jumbo clusters in use (current/cache/total/max)
567K/375K/942K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
0 calls to protocol drain routines

------------------------------------------------------------------------
netstat -id

Name    Mtu Network       Address              Ipkts Ierrs Idrop    Opkts Oerrs  Coll Drop
em0    1500 <Link#1>      00:30:48:2d:32:6a       23     0     0        8     0     0    0 
em0    1500 172.16.8.0    freebeast                8     -     -        5     -     -    - 
em0    1500 fe80:1::230:4 fe80:1::230:48ff:        0     -     -        1     -     -    - 
em1*   1500 <Link#2>      00:30:48:2d:32:6b        0     0     0        0     0     0    0 
lo0   16384 <Link#3>                               4     0     0        0     0     0    0 
lo0   16384 your-net      localhost                0     -     -        0     -     -    - 
lo0   16384 localhost     ::1                      0     -     -        0     -     -    - 
lo0   16384 fe80:3::1     fe80:3::1                0     -     -        0     -     -    - 

------------------------------------------------------------------------
netstat -anr

Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            172.16.8.1         UGS         0        0    em0
127.0.0.1          link#3             UH          0        0    lo0
172.16.8.0/24      link#1             U           4        5    em0
172.16.8.10        link#1             UHS         0        0    lo0

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               ::1                           UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%em0/64                     link#1                        U           em0
fe80::230:48ff:fe2d:326a%em0      link#1                        UHS         lo0
fe80::%lo0/64                     link#3                        U           lo0
fe80::1%lo0                       link#3                        UHS         lo0
ff01:1::/32                       fe80::230:48ff:fe2d:326a%em0  U           em0
ff01:3::/32                       ::1                           U           lo0
ff02::/16                         ::1                           UGRS        lo0
ff02::%em0/32                     fe80::230:48ff:fe2d:326a%em0  U           em0
ff02::%lo0/32                     ::1                           U           lo0

------------------------------------------------------------------------
netstat -anA

Active Internet connections (including servers)
Tcpcb    Proto Recv-Q Send-Q  Local Address      Foreign Address   (state)
c69b94f0 tcp4       0      0 *.925              *.*                LISTEN
c69b2c58 tcp4       0      0 *.111              *.*                LISTEN
c69b3000 tcp6       0      0 *.111              *.*                LISTEN
c686c528 udp4       0      0 *.907              *.*                
c686b7bc udp6       0      0 *.*                *.*                
c686b000 udp4       0      0 *.832              *.*                
c686b1b8 udp4       0      0 *.111              *.*                
c686b294 udp6       0      0 *.736              *.*                
c686b44c udp6       0      0 *.111              *.*                
c686b604 udp4       0      0 *.514              *.*                
c686b528 udp6       0      0 *.514              *.*                
Active UNIX domain sockets
Address  Type   Recv-Q Send-Q    Inode     Conn     Refs  Nextref Addr
c6942ac0 stream      0      0 c69af440        0        0        0 /var/run/rpcbind.sock
c6942d70 stream      0      0 c6944660        0        0        0 /var/run/devd.pipe
c6943e1c dgram       0      0 c69ac440        0        0        0 /var/run/logpriv
c6943ec8 dgram       0      0 c6861880        0        0        0 /var/run/log

------------------------------------------------------------------------
netstat -aL

Current listen queue sizes (qlen/incqlen/maxqlen)
Proto Listen         Local Address         
tcp4  0/0/128        *.925                  
tcp4  0/0/128        *.sunrpc               
tcp6  0/0/128        *.sunrpc               
unix  0/0/128        /var/run/rpcbind.sock
unix  0/0/4          /var/run/devd.pipe

------------------------------------------------------------------------
fstat

USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
root     sh           921 root /             2 drwxr-xr-x    1024  r
root     sh           921   wd /             2 drwxr-xr-x    1024  r
root     sh           921 text /        119072 -r-xr-xr-x  120188  r
root     sh           921    0 /dev          5 crw-------  console rw
root     sh           921    1 /var     494633 -rw-r--r--       0  w
root     sh           921    2 /dev         25 crw-rw-rw-    null  w
root     sh           915 root /             2 drwxr-xr-x    1024  r
root     sh           915   wd /             2 drwxr-xr-x    1024  r
root     sh           915 text /        119072 -r-xr-xr-x  120188  r
root     sh           915    0 /dev          5 crw-------  console rw
root     sh           915    1 /dev          5 crw-------  console rw
root     sh           915    2 /dev          5 crw-------  console rw
root     sh           915   10 /         47279 -r-xr-xr-x     972  r
root     ypbind       908 root /             2 drwxr-xr-x    1024  r
root     ypbind       908   wd /             2 drwxr-xr-x    1024  r
root     ypbind       908 text /usr     286756 -r-xr-xr-x   16276  r
root     ypbind       908    0 /dev         25 crw-rw-rw-    null rw
root     ypbind       908    1 /dev         25 crw-rw-rw-    null rw
root     ypbind       908    2 /dev         25 crw-rw-rw-    null rw
root     ypbind       908    3 /var     494632 -r--r--r--       0  r
root     ypbind       908    4* internet dgram udp c686c528
root     ypbind       908    5* internet stream tcp c69b94f0
root     ypbind       908    6 /var      94209 -rw-r--r--      14 rw
root     rpcbind      881 root /             2 drwxr-xr-x    1024  r
root     rpcbind      881   wd /             2 drwxr-xr-x    1024  r
root     rpcbind      881 text /usr     285496 -r-xr-xr-x   39500  r
root     rpcbind      881    0 /dev         25 crw-rw-rw-    null rw
root     rpcbind      881    1 /dev         25 crw-rw-rw-    null rw
root     rpcbind      881    2 /dev         25 crw-rw-rw-    null rw
root     rpcbind      881    3 /var     494629 -r--r--r--       0  r
root     rpcbind      881    4* internet6 dgram udp c686b7bc
root     rpcbind      881    5* local stream c6942ac0
root     rpcbind      881    6* internet6 dgram udp c686b44c
root     rpcbind      881    7* internet6 dgram udp c686b294
root     rpcbind      881    8* internet6 stream tcp c69b3000
root     rpcbind      881    9* internet dgram udp c686b1b8
root     rpcbind      881   10* internet dgram udp c686b000
root     rpcbind      881   11* internet stream tcp c69b2c58
root     syslogd      857 root /             2 drwxr-xr-x    1024  r
root     syslogd      857   wd /             2 drwxr-xr-x    1024  r
root     syslogd      857 text /usr     286269 -r-xr-xr-x   35860  r
root     syslogd      857    0 /dev         25 crw-rw-rw-    null rw
root     syslogd      857    1 /dev         25 crw-rw-rw-    null rw
root     syslogd      857    2 /dev         25 crw-rw-rw-    null rw
root     syslogd      857    3 /var     494624 -rw-------       3  w
root     syslogd      857    4* local dgram c6943ec8
root     syslogd      857    5* local dgram c6943e1c
root     syslogd      857    6* internet6 dgram udp c686b528
root     syslogd      857    7* internet dgram udp c686b604
root     syslogd      857    8 /dev         28 crw-------    klog  r
root     syslogd      857   10 /dev          5 crw-------  console  w
root     syslogd      857   11 /var     141316 -rw-r--r--  103943  w
root     syslogd      857   12 /var     141323 -rw-------      55  w
root     syslogd      857   13 /var     141317 -rw-------   81022  w
root     syslogd      857   14 /var     141315 -rw-r-----    3264  w
root     syslogd      857   15 /var     141319 -rw-r--r--   64145  w
root     syslogd      857   16 /var     141325 -rw-------      55  w
root     syslogd      857   17 /var     141320 -rw-------   26693  w
root     syslogd      857   18 /var     141318 -rw-------      55  w
root     syslogd      857   19 /var     141322 -rw-r-----      55  w
root     devd         660 root /             2 drwxr-xr-x    1024  r
root     devd         660   wd /             2 drwxr-xr-x    1024  r
root     devd         660 text /            75 -r-xr-xr-x  400520  r
root     devd         660    0 /dev         25 crw-rw-rw-    null rw
root     devd         660    1 /dev         25 crw-rw-rw-    null rw
root     devd         660    2 /dev         25 crw-rw-rw-    null rw
root     devd         660    3 /         23572 drwxr-xr-x     512  r
root     devd         660    4 /dev          4 crw-------  devctl  r
root     devd         660    5* local stream c6942d70
root     devd         660    6 /var     494620 -rw-------       3  w
root     sh            21 root /             2 drwxr-xr-x    1024  r
root     sh            21   wd /             2 drwxr-xr-x    1024  r
root     sh            21 text /        119072 -r-xr-xr-x  120188  r
root     sh            21    0 /dev          5 crw-------  console rw
root     sh            21    1 /dev          5 crw-------  console rw
root     sh            21    2 /dev          5 crw-------  console rw
root     sh            21   10 /         23873 -rw-r--r--    3710  r
root     init           1 root /             2 drwxr-xr-x    1024  r
root     init           1   wd /             2 drwxr-xr-x    1024  r
root     init           1 text /            91 -r-xr-xr-x  677768  r
root     kernel         0 root /             2 drwxr-xr-x    1024  r
root     kernel         0   wd /             2 drwxr-xr-x    1024  r

------------------------------------------------------------------------
dmesg

Copyright (c) 1992-2010 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-CURRENT #155 r207844M: Mon May 10 14:55:41 PDT 2010
    root_at_freebeast.catwhisker.org:/usr/obj/usr/src/sys/GENERIC i386
WARNING: WITNESS option enabled, expect reduced performance.
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 3.60GHz (3610.95-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf41  Family = f  Model = 4  Stepping = 1
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x659d<SSE3,DTES64,MON,DS_CPL,EST,TM2,CNXT-ID,CX16,xTPR>
  AMD Features=0x20100000<NX,LM>
  TSC: P-state invariant
real memory  = 2147483648 (2048 MB)
avail memory = 2086129664 (1989 MB)
ACPI APIC Table: <PTLTD  	 APIC  >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 2 package(s) x 1 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  6
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic2 <Version 2.0> irqs 48-71 on motherboard
kbd1 at kbdmux0
acpi0: <PTLTD   RSDT> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <unknown> at device 0.1 (no driver attached)
pci0: <base peripheral> at device 1.0 (no driver attached)
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 0.0 on pci1
pci2: <ACPI PCI bus> on pcib2
aac0: <Adaptec SCSI RAID 2200S> mem 0xdc000000-0xdfffffff irq 24 at device 1.0 on pci2
aac0: Enable Raw I/O
aac0: New comm. interface enabled
aac0: [ITHREAD]
aac0: Adaptec 2200S, aac driver 2.1.9-1
aacp0: <SCSI Passthrough Bus> on aac0
aacp1: <SCSI Passthrough Bus> on aac0
pcib3: <ACPI PCI-PCI bridge> at device 0.2 on pci1
pci3: <ACPI PCI bus> on pcib3
em0: <Intel(R) PRO/1000 Legacy Network Connection 1.0.1> port 0x2000-0x203f mem 0xd8200000-0xd821ffff irq 54 at device 2.0 on pci3
em0: [FILTER]
em0: Ethernet address: 00:30:48:2d:32:6a
em1: <Intel(R) PRO/1000 Legacy Network Connection 1.0.1> port 0x2040-0x207f mem 0xd8220000-0xd823ffff irq 55 at device 2.1 on pci3
em1: [FILTER]
em1: Ethernet address: 00:30:48:2d:32:6b
pcib4: <ACPI PCI-PCI bridge> irq 16 at device 4.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> irq 16 at device 6.0 on pci0
pci5: <ACPI PCI bus> on pcib5
uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0x1400-0x141f irq 16 at device 29.0 on pci0
uhci0: [ITHREAD]
usbus0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0x1420-0x143f irq 19 at device 29.1 on pci0
uhci1: [ITHREAD]
usbus1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
uhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0x1440-0x145f irq 18 at device 29.2 on pci0
uhci2: [ITHREAD]
usbus2: <Intel 82801EB (ICH5) USB controller USB-C> on uhci2
uhci3: <Intel 82801EB (ICH5) USB controller USB-D> port 0x1460-0x147f irq 16 at device 29.3 on pci0
uhci3: [ITHREAD]
usbus3: <Intel 82801EB (ICH5) USB controller USB-D> on uhci3
ehci0: <Intel 82801EB/R (ICH5) USB 2.0 controller> mem 0xd8001000-0xd80013ff irq 23 at device 29.7 on pci0
ehci0: [ITHREAD]
usbus4: EHCI version 1.0
usbus4: <Intel 82801EB/R (ICH5) USB 2.0 controller> on ehci0
pcib6: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci6: <ACPI PCI bus> on pcib6
vgapci0: <VGA-compatible display> port 0x3000-0x30ff mem 0xd9000000-0xd9ffffff,0xd8300000-0xd8300fff irq 17 at device 1.0 on pci6
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH5 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x14a0-0x14af at device 31.1 on pci0
ata0: <ATA channel 0> on atapci0
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci0
ata1: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_button0: <Power Button> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model Generic PS/2 mouse, device ID 0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
uart0: console (9600,n,8,1)
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart1: [FILTER]
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FILTER]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
pmtimer0 on isa0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0xc9fff,0xca000-0xcafff,0xcb000-0xcf7ff pnpid ORM0000 on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: parallel port not found.
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 122d00000e24
device_attach: est0 attach returned 6
p4tcc0: <CPU Frequency Thermal Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 122d00000e24
device_attach: est1 attach returned 6
p4tcc1: <CPU Frequency Thermal Control> on cpu1
Timecounters tick every 1.000 msec
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 480Mbps High Speed USB v2.0
ata1: DMA limited to UDMA33, controller found non-ATA66 cable
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen4.1: <Intel> at usbus4
uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
acd0: DVDROM <MATSHITADVD-ROM SR-8178/PZ16> at ata1-slave UDMA33 
aacd0: <RAID 1 (Mirror)> on aac0
aacd0: 34970MB (71619584 sectors)
aacd1: <RAID 1 (Mirror)> on aac0
aacd1: 69974MB (143307008 sectors)
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
uhub4: 8 ports with 8 removable, self powered
ses0 at aacp0 bus 0 scbus0 target 6 lun 0
ses0: <SUPER GEM318 0> Fixed Uninstalled SCSI-2 device 
ses0: 3.300MB/s transfers
ses0: SAF-TE Compliant Device
pass0 at aacp0 bus 0 scbus0 target 0 lun 0
pass0: <SEAGATE ST336754LC 0003> Fixed Uninstalled SCSI-3 device 
pass0: 3.300MB/s transfers
pass1 at aacp0 bus 0 scbus0 target 1 lun 0
pass1: <SEAGATE ST336754LC 0003> Fixed Uninstalled SCSI-3 device 
pass1: 3.300MB/s transfers
pass2 at aacp0 bus 0 scbus0 target 2 lun 0
pass2: <SEAGATE ST373454LC 0005> Fixed Uninstalled SCSI-3 device 
pass2: 3.300MB/s transfers
pass3 at aacp0 bus 0 scbus0 target 3 lun 0
pass3: <SEAGATE ST373454LC 0005> Fixed Uninstalled SCSI-3 device 
pass3: 3.300MB/s transfers
SMP: AP CPU #1 Launched!
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/aacd0s4a
Setting hostuuid: 80f1e964-dc63-0010-89d5-0030482d326a.
Setting hostid: 0xc74551dd.
Entropy harvesting:
 interrupts
 ethernet
 point_to_point
 kickstart
.
Starting file system checks:
/dev/aacd0s4a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s4a: clean, 422048 free (1216 frags, 52604 blocks, 0.2% fragmentation)
/dev/aacd1s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd1s1e: clean, 4681662 free (261046 frags, 552577 blocks, 2.0% fragmentation)
/dev/aacd0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s1a: clean, 424969 free (1913 frags, 52882 blocks, 0.3% fragmentation)
/dev/aacd0s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s1d: clean, 1031838 free (4190 frags, 128456 blocks, 0.2% fragmentation)
/dev/aacd1s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd1s1d: clean, 11717829 free (197893 frags, 1439992 blocks, 1.2% fragmentation)
/dev/aacd0s2a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s2a: clean, 596256 free (1488 frags, 74346 blocks, 0.2% fragmentation)
/dev/aacd0s2d: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s2d: clean, 1108521 free (23105 frags, 135677 blocks, 1.3% fragmentation)
/dev/aacd0s3a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s3a: clean, 578034 free (1146 frags, 72111 blocks, 0.2% fragmentation)
/dev/aacd0s3d: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s3d: clean, 1070844 free (39380 frags, 128933 blocks, 2.2% fragmentation)
/dev/aacd0s4d: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s4d: clean, 1075950 free (60166 frags, 126973 blocks, 3.4% fragmentation)
/dev/aacd0s4f: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s4f: clean, 2106304 free (344 frags, 263245 blocks, 0.0% fragmentation)
Mounting local file systems:
.
Setting hostname: freebeast.catwhisker.org
.
lock order reversal:
 1st 0xc683e7c8 ufs (ufs) _at_ /usr/src/sys/kern/vfs_lookup.c:502
 2nd 0xd94fe700 bufwait (bufwait) _at_ /usr/src/sys/ufs/ffs/ffs_softdep.c:11363
 3rd 0xc6871168 ufs (ufs) _at_ /usr/src/sys/kern/vfs_subr.c:2091
KDB: stack backtrace:
db_trace_self_wrapper(c0cc72a5,ebdf2300,c08e9f65,c08da2db,c0cca298,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c08da2db,c0cca298,c553d030,c5540568,ebdf235c,...) at kdb_backtrace+0x29
_witness_debugger(c0cca298,c6871168,c0cbc5bd,c5540568,c0cd13b1,...) at _witness_debugger+0x25
witness_checkorder(c6871168,9,c0cd13b1,82b,0,...) at witness_checkorder+0x839
__lockmgr_args(c6871168,80100,c6871188,0,0,...) at __lockmgr_args+0x7f9
ffs_lock(ebdf2480,c08e9d0b,c0cd0856,80100,c6871110,...) at ffs_lock+0x8a
VOP_LOCK1_APV(c0dd4ee0,ebdf2480,c63cecd4,c0def920,c6871110,...) at VOP_LOCK1_APV+0xb5
_vn_lock(c6871110,80100,c0cd13b1,82b,4,...) at _vn_lock+0x5e
vget(c6871110,80100,c63cec30,50,0,...) at vget+0xb9
vfs_hash_get(c6857ca8,78c22,80000,c63cec30,ebdf25d0,...) at vfs_hash_get+0xe6
ffs_vgetf(c6857ca8,78c22,80000,ebdf25d0,1,...) at ffs_vgetf+0x49
softdep_sync_metadata(c683e770,0,c0ced5f8,144,0,...) at softdep_sync_metadata+0xc92
ffs_syncvnode(c683e770,1,c63cec30,ebdf2690,246,...) at ffs_syncvnode+0x3e2
ffs_truncate(c683e770,200,0,880,c5588480,...) at ffs_truncate+0x862
ufs_direnter(c683e770,c6871110,ebdf2944,ebdf2bd4,0,...) at ufs_direnter+0x8d4
ufs_makeinode(ebdf2bd4,0,ebdf2b30,ebdf2a8c,c0c032e5,...) at ufs_makeinode+0x557
ufs_create(ebdf2b30,ebdf2b48,0,0,ebdf2ba8,...) at ufs_create+0x30
VOP_CREATE_APV(c0dd4ee0,ebdf2b30,ebdf2bd4,ebdf2ac8,0,...) at VOP_CREATE_APV+0xa5
vn_open_cred(ebdf2ba8,ebdf2c5c,1a4,0,c5588480,...) at vn_open_cred+0x215
vn_open(ebdf2ba8,ebdf2c5c,1a4,c637ee70,0,...) at vn_open+0x3b
kern_openat(c63cec30,ffffff9c,804c5e8,0,602,...) at kern_openat+0x125
kern_open(c63cec30,804c5e8,0,601,21b6,...) at kern_open+0x35
open(c63cec30,ebdf2cf8,c0cffdfb,c0caa6d4,c63c9d48,...) at open+0x30
syscall(ebdf2d38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (5, FreeBSD ELF32, open), eip = 0x281744b3, esp = 0xbfbfec4c, ebp = 0xbfbfecb8 ---
Starting Network: lo0 em0 em1.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=3<RXCSUM,TXCSUM>
	inet 127.0.0.1 netmask 0xff000000 
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
	ether 00:30:48:2d:32:6a
	inet 172.16.8.10 netmask 0xffffff00 broadcast 172.16.8.255
	inet6 fe80::230:48ff:fe2d:326a%em0 prefixlen 64 tentative scopeid 0x1 
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect
	status: no carrier
em1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
	ether 00:30:48:2d:32:6b
	media: Ethernet autoselect
	status: no carrier
Starting devd.
Starting Network: em1.
em1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
	ether 00:30:48:2d:32:6b
	media: Ethernet autoselect
	status: no carrier
add net default: gateway 172.16.8.1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
add net fe80::: gateway ::1
add net ff02::: gateway ::1
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat
a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout
Creating and/or trimming log files
.
Starting syslogd.
No core dumps found.
Starting rpcbind.
NFS access cache time=60
em0: link state changed to UP
Setting NIS domain: lmdhw.com.
Starting ypbind.
Starting amd.
Kernel page fault with the following non-sleepable locks held:
exclusive sleep mutex mld_mtx (mld_mtx) r = 0 (0xc0f96468) locked _at_ /usr/src/sys/netinet6/mld6.c:1352
exclusive sleep mutex in6_multi_mtx (in6_multi_mtx) r = 0 (0xc0f95408) locked _at_ /usr/src/sys/netinet6/mld6.c:1351
KDB: stack backtrace:
db_trace_self_wrapper(c0cc72a5,c5213864,c08e9f65,c0ce3e07,547,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0ce3e07,547,ffffffff,c0f65a1c,c521389c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc982b,c52138b0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cffec8,c55810a4,c557f7f8,...) at witness_warn+0x1fd
trap(c521393c) at trap+0x19e
calltrap() at calltrap+0x6
--- trap 0xc, eip = 0xc0a4d340, esp = 0xc521397c, ebp = 0xc5213b14 ---
ip6_output(c5a4fc00,c0f964a0,0,1,c5213b44,...) at ip6_output+0x840
mld_dispatch_packet(0,0,c5213c1c,c0a509a7,c56fca2c,...) at mld_dispatch_packet+0x30d
mld_dispatch_queue(c56fca2c,0,c0ce3e07,591,c0cbb984,...) at mld_dispatch_queue+0x38
mld_fasttimo(c5213c48,c0904cc9,c5213c48,c0893aa4,c0e24f80,...) at mld_fasttimo+0x747
icmp6_fasttimo(c5213c48,c0893aa4,c0e24f80,4,c0cc24d6,...) at icmp6_fasttimo+0x8
pffasttimo(0,0,c0cc5113,189,c0e24fb8,...) at pffasttimo+0x29
softclock(c0e24f80,c5213cc8,c0893aa4,c0e28940,c5588838,...) at softclock+0x24a
intr_event_execute_handlers(c557f7f8,c5588800,c0cbf33f,533,c5588870,...) at intr_event_execute_handlers+0x125
ithread_loop(c557e120,c5213d38,c0cbf0b8,343,c557f7f8,...) at ithread_loop+0x9f
fork_exit(c087c780,c557e120,c5213d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xc5213d70, ebp = 0 ---


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address	= 0x3c
fault code		= supervisor read, page not present
instruction pointer	= 0x20:0xc0a4d340
stack pointer	        = 0x28:0xc521397c
frame pointer	        = 0x28:0xc5213b14
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, def32 1, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 12 (swi4: clock)
panic: from debugger
cpuid = 0
Uptime: 20s
Physical memory: 2030 MB
Dumping 89 MB: 74 58 42 26 10

------------------------------------------------------------------------
kernel config

options	CONFIG_AUTOGENERATED
ident	GENERIC
machine	i386
cpu	I686_CPU
cpu	I586_CPU
cpu	I486_CPU
makeoptions	DEBUG=-g
options	USB_DEBUG
options	AH_SUPPORT_AR5416
options	IEEE80211_SUPPORT_MESH
options	IEEE80211_AMPDU_AGE
options	IEEE80211_DEBUG
options	AHD_REG_PRETTY_PRINT
options	AHC_REG_PRETTY_PRINT
options	ATA_STATIC_ID
options	SMP
options	WITNESS_SKIPSPIN
options	WITNESS
options	INVARIANT_SUPPORT
options	INVARIANTS
options	DEADLKRES
options	GDB
options	DDB
options	KDB
options	INCLUDE_CONFIG_FILE
options	FLOWTABLE
options	MAC
options	AUDIT
options	HWPMC_HOOKS
options	KBD_INSTALL_CDEV
options	PRINTF_BUFR_SIZE=128
options	_KPOSIX_PRIORITY_SCHEDULING
options	P1003_1B_SEMAPHORES
options	SYSVSEM
options	SYSVMSG
options	SYSVSHM
options	STACK
options	KTRACE
options	SCSI_DELAY=5000
options	COMPAT_FREEBSD7
options	COMPAT_FREEBSD6
options	COMPAT_FREEBSD5
options	COMPAT_FREEBSD4
options	GEOM_LABEL
options	GEOM_PART_GPT
options	PSEUDOFS
options	PROCFS
options	CD9660
options	MSDOSFS
options	NFS_ROOT
options	NFSLOCKD
options	NFSSERVER
options	NFSCLIENT
options	MD_ROOT
options	UFS_GJOURNAL
options	UFS_DIRHASH
options	UFS_ACL
options	SOFTUPDATES
options	FFS
options	SCTP
options	INET6
options	INET
options	PREEMPTION
options	SCHED_ULE
options	NATIVE
options	GEOM_PART_MBR
options	GEOM_PART_EBR_COMPAT
options	GEOM_PART_EBR
options	GEOM_PART_BSD
options	ISAPNP
device	isa
device	npx
device	mem
device	io
device	uart_ns8250
device	atpic
device	apic
device	cpufreq
device	acpi
device	eisa
device	pci
device	fdc
device	ata
device	atadisk
device	ataraid
device	atapicd
device	atapifd
device	atapist
device	ahb
device	ahc
device	ahd
device	amd
device	hptiop
device	isp
device	mpt
device	sym
device	trm
device	adv
device	adw
device	aha
device	aic
device	bt
device	ncv
device	nsp
device	stg
device	scbus
device	ch
device	da
device	sa
device	cd
device	pass
device	ses
device	amr
device	arcmsr
device	asr
device	ciss
device	dpt
device	hptmv
device	hptrr
device	iir
device	ips
device	mly
device	twa
device	aac
device	aacp
device	ida
device	mfi
device	mlx
device	pst
device	twe
device	atkbdc
device	atkbd
device	psm
device	kbdmux
device	vga
device	splash
device	sc
device	agp
device	pmtimer
device	cbb
device	pccard
device	cardbus
device	uart
device	ppc
device	ppbus
device	lpt
device	plip
device	ppi
device	de
device	em
device	igb
device	ixgb
device	le
device	ti
device	txp
device	vx
device	miibus
device	ae
device	age
device	alc
device	ale
device	bce
device	bfe
device	bge
device	dc
device	et
device	fxp
device	jme
device	lge
device	msk
device	nfe
device	nge
device	pcn
device	re
device	rl
device	sf
device	sge
device	sis
device	sk
device	ste
device	stge
device	tl
device	tx
device	vge
device	vr
device	wb
device	xl
device	cs
device	ed
device	ex
device	ep
device	fe
device	ie
device	sn
device	xe
device	wlan
device	wlan_wep
device	wlan_ccmp
device	wlan_tkip
device	wlan_amrr
device	an
device	ath
device	ath_hal
device	ath_rate_sample
device	ral
device	wi
device	loop
device	random
device	ether
device	vlan
device	tun
device	pty
device	md
device	gif
device	faith
device	firmware
device	bpf
device	uhci
device	ohci
device	ehci
device	usb
device	uhid
device	ukbd
device	ulpt
device	umass
device	ums
device	urio
device	u3g
device	uark
device	ubsa
device	uftdi
device	uipaq
device	uplcom
device	uslcom
device	uvisor
device	uvscom
device	aue
device	axe
device	cdce
device	cue
device	kue
device	rue
device	udav
device	rum
device	uath
device	ural
device	zyd
device	firewire
device	sbp
device	fwe
device	fwip
device	dcons
device	dcons_crom

------------------------------------------------------------------------
ddb capture buffer


Peace,
david
-- 
David H. Wolfskill				david_at_catwhisker.org
Depriving a girl or boy of an opportunity for education is evil.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.

Received on Mon May 10 2010 - 20:52:09 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:03 UTC