Once a week or so we're seeing a panic with a -current kernel built just before the gcc 4.2 import (maybe three weeks ago). The box has a Supermicro X7DBE/X7DBE+ motherboard with two Xeon 5160s, 16G RAM, and an Areca 1220 controller with eight 500G disks connected. Does this indicate that the arcmsr driver is at fault: Tracing command irq16: arcmsr0 pid 26 tid 100018 td 0xffffff040fc5b000 cpustop_handler() at cpustop_handler+0x35 ipi_nmi_handler() at ipi_nmi_handler+0x2e trap() at trap+0x365 nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff8041ab11, rsp = 0xffffffffab59eff0, rbp = 0xffffffffac0a37d0 --- siocnclose() at siocnclose+0x21 sio_cnputc() at sio_cnputc+0x89 cnputc() at cnputc+0x6a putchar() at putchar+0x5f kvprintf() at kvprintf+0xd45 printf() at printf+0xe1 panic() at panic+0x145 xpt_done() at xpt_done+0x14a arcmsr_interrupt() at arcmsr_interrupt+0x2df ithread_loop() at ithread_loop+0x108 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0a3d30, rbp = 0 --- Following is a bunch of ddb output and dmesg. The initial KDB backtrace seems to have a panic message intermingled somehow. It looks like the arcmsr driver caused a panic, which in turn fails because of a lock violation. What else can I do to track down the problem? TIA, Matt KDB: stack backtrace: pandb_trace_self_wrapper() at idb_trace_self_wrapper+0x3a c: vfs_badlock() at Bvfs_badlock+0x95 adassert_vop_locked() at assert_vop_locked+0x77 taiVOP_READ_APV() at lVOP_READ_APV+0x8a q Nvn_read() at Evn_read+0x244 XTdofileread() at (dofileread+0x94 0xkern_readv() at fkern_readv+0x60 ffread() at fread+0x4a ff0syscall() at 0syscall+0x20b 00Xfast_syscall() at aXfast_syscall+0xab 5--- syscall (3d, FreeBSD ELF64, read), rip = 0x800c7aa2c, rsp = 0x7fffffffc758, rbp = 0 --- 6VOP_READ: 0xffffff0336319798 is not locked but should be KDB: enter: lock violation 2[thread pid 66384 tid 100138 ] Stopped at kdb_enter+0x2f: nop db> trace Tracing pid 66384 tid 100138 td 0xffffff0345b6f2c0 kdb_enter() at kdb_enter+0x2f assert_vop_locked() at assert_vop_locked+0x77 VOP_READ_APV() at VOP_READ_APV+0x8a vn_read() at vn_read+0x244 dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800c7aa2c, rsp = 0x7fffffffc758, rbp = 0 --- db> ps pid ppid pgrp uid state wmesg wchan cmd 66384 66292 687 1003 R CPU 1 perl5.8.8 66383 687 687 80 S lockf 0xffffff030c65d280 httpd 66331 687 687 80 S select 0xffffffff8069a830 httpd 66292 687 687 80 S kqread 0xffffff02afe39600 httpd 66290 687 687 80 S select 0xffffffff8069a830 httpd 66288 687 687 80 S select 0xffffffff8069a830 httpd 66279 687 687 80 S select 0xffffffff8069a830 httpd 66277 687 687 80 S select 0xffffffff8069a830 httpd 66275 687 687 80 S select 0xffffffff8069a830 httpd 66274 687 687 80 S kqread 0xffffff02c8c1b700 httpd 66267 687 687 80 S select 0xffffffff8069a830 httpd 66210 687 687 80 S lockf 0xffffff02d96ba580 httpd 66206 687 687 80 S select 0xffffffff8069a830 httpd 66203 687 687 80 S select 0xffffffff8069a830 httpd 66031 687 687 80 S select 0xffffffff8069a830 httpd 53467 53465 53465 1003 S select 0xffffffff8069a830 sftp-server 53465 53464 53465 1003 Ss pause 0xffffff024debbac8 csh 53464 53462 53462 1003 S select 0xffffffff8069a830 sshd 53462 699 53462 0 Ss sbwait 0xffffff03b3932984 sshd 3194 77246 3194 0 S+ select 0xffffffff8069a830 ssh 69351 77246 69351 0 T+ emacs 38702 77246 38702 0 T+ ping 37157 37146 37157 0 S+ ttyin 0xffffff0000a5a010 bash 37146 37142 37146 1005 S+ wait 0xffffff02db456000 su 37142 37141 37142 1005 Ss+ wait 0xffffff030b361380 bash 37141 37123 37123 1005 S select 0xffffffff8069a830 sshd 37123 699 37123 0 Ss sbwait 0xffffff034d8a0c3c sshd 77253 77246 77253 0 T+ emacs 77246 77235 77246 0 S+ wait 0xffffff034cb40000 bash 77235 77054 77235 1001 S+ wait 0xffffff004ba44380 su 77054 77001 77054 1001 S+ wait 0xffffff02cff28700 bash 77001 77000 77001 1001 Ss+ wait 0xffffff007b41aa80 sh 77000 76971 76971 1001 S select 0xffffffff8069a830 sshd 76971 699 76971 0 Ss sbwait 0xffffff02e3cc4c3c sshd 831 830 831 1005 Ss+ ttyin 0xffffff00009f7010 bash 830 827 827 1005 S select 0xffffffff8069a830 sshd 827 699 827 0 Ss sbwait 0xffffff0349f216cc sshd 765 1 765 0 Ss+ ttyin 0xffffff0000a84810 getty 764 1 764 0 Ss+ ttyin 0xffffff0000a82410 getty 763 1 763 0 Ss+ ttyin 0xffffff0000b00410 getty 762 1 762 0 Ss+ ttyin 0xffffff0000a84410 getty 761 1 761 0 Ss+ ttyin 0xffffff040fa5c410 getty 760 1 760 0 Ss+ ttyin 0xffffff0000abfc10 getty 759 1 759 0 Ss+ ttyin 0xffffff0000abf810 getty 758 1 758 0 Ss+ ttyin 0xffffff0000abe810 getty 757 1 757 0 Ss+ ttyin 0xffffff0000abe410 getty 742 1 742 0 Ss select 0xffffffff8069a830 inetd 715 1 715 0 Ss nanslp 0xffffffff80609bc8 cron 709 1 709 25 Ss pause 0xffffff008bb63ac8 sendmail 705 1 705 0 Ss select 0xffffffff8069a830 sendmail 699 1 699 0 Ss select 0xffffffff8069a830 sshd 687 1 687 0 Ss nanslp 0xffffffff80609bc8 httpd 675 644 643 88 S+ (threaded) mysqld 100283 S ucond 0xffffff003fe30800 mysqld 100282 S ucond 0xffffff0000afc580 mysqld 100281 S ucond 0xffffff03059b9900 mysqld 100264 S ucond 0xffffff02c9816680 mysqld 100103 S sigwait 0xffffffffae6b69f8 mysqld 100100 S ucond 0xffffff01151a9900 mysqld 100102 S select 0xffffffff8069a830 mysqld 100101 S select 0xffffffff8069a830 mysqld 100099 S ucond 0xffffff01151a9a00 mysqld 100098 S ucond 0xffffff0000b15c00 mysqld 100097 S ucond 0xffffff0000b15d00 mysqld 100096 S ucond 0xffffff0000d1d100 mysqld 100091 S select 0xffffffff8069a830 initial thread 644 1 643 88 S+ wait 0xffffff010b0b1000 sh 536 1 536 0 Ss select 0xffffffff8069a830 syslogd 469 1 469 0 Ss select 0xffffffff8069a830 devd 110 1 110 0 Ss pause 0xffffff006635c748 adjkerntz 38 0 0 0 SL sdflush 0xffffffff806a9308 [softdepflush] 37 0 0 0 SL syncer 0xffffffff80609840 [syncer] 36 0 0 0 SL vlruwt 0xffffff040fbfe000 [vnlru] 35 0 0 0 SL psleep 0xffffffff8069b09c [bufdaemon] 34 0 0 0 SL pgzero 0xffffffff806ba2a0 [pagezero] 33 0 0 0 SL pollid 0xffffffff8060898c [idlepoll] 32 0 0 0 SL psleep 0xffffffff806aa14c [vmdaemon] 31 0 0 0 SL psleep 0xffffffff806aa104 [pagedaemon] 30 0 0 0 WL [swi0: sio] 29 0 0 0 WL [irq1: atkbd0] 28 0 0 0 WL [irq15: ata1] 27 0 0 0 WL [irq14: ata0] 26 0 0 0 RL CPU 2 [irq16: arcmsr0] 25 0 0 0 WL [irq19: em1] 24 0 0 0 RL [irq18: em0] 23 0 0 0 WL [irq9: acpi0] 22 0 0 0 SL - 0xffffff00009d2580 [acpi_task_2] 9 0 0 0 SL - 0xffffff00009d2580 [acpi_task_1] 8 0 0 0 SL - 0xffffff00009d2580 [acpi_task_0] 7 0 0 0 SL - 0xffffff00009d9a80 [kqueue taskq] 21 0 0 0 WL [swi6: task queue] 20 0 0 0 WL [swi6: Giant taskq] 6 0 0 0 SL - 0xffffff00009d9d00 [thread taskq] 19 0 0 0 WL [swi5: +] 18 0 0 0 RL [swi2: cambio] 5 0 0 0 SL ccb_scan 0xffffffff805ef440 [xpt_thrd] 17 0 0 0 SL - 0xffffffff80609848 [yarrow] 4 0 0 0 SL - 0xffffffff80605b18 [g_down] 3 0 0 0 SL - 0xffffffff80605b10 [g_up] 2 0 0 0 SL - 0xffffffff80605b00 [g_event] 16 0 0 0 WL [swi1: net] 15 0 0 0 WL [swi3: vm] 14 0 0 0 RL [swi4: clock sio] 13 0 0 0 RL CPU 0 [idle: cpu0] 12 0 0 0 RL [idle: cpu1] 11 0 0 0 RL [idle: cpu2] 10 0 0 0 RL CPU 3 [idle: cpu3] 1 0 1 0 SLs wait 0xffffff040fc98000 [init] 0 0 0 0 WLs [swapper] 66360 66331 687 1003 Z perl5.8.8 db> show pcpu cpuid = 1 curthread = 0xffffff0345b6f2c0: pid 66384 "perl5.8.8" curpcb = 0xffffffffae76fd40 fpcurthread = 0xffffff0345b6f2c0: pid 66384 "perl5.8.8" idlethread = 0xffffff040fc93b00: pid 12 "idle: cpu1" spin locks held: db> show allpcpu Current CPU: 1 cpuid = 0 curthread = 0xffffff040fc56000: pid 13 "idle: cpu0" curpcb = 0xffffffffac06fd40 fpcurthread = none idlethread = 0xffffff040fc56000: pid 13 "idle: cpu0" spin locks held: cpuid = 1 curthread = 0xffffff0345b6f2c0: pid 66384 "perl5.8.8" curpcb = 0xffffffffae76fd40 fpcurthread = 0xffffff0345b6f2c0: pid 66384 "perl5.8.8" idlethread = 0xffffff040fc93b00: pid 12 "idle: cpu1" spin locks held: cpuid = 2 curthread = 0xffffff040fc5b000: pid 26 "irq16: arcmsr0" curpcb = 0xffffffffac0a3d40 fpcurthread = none idlethread = 0xffffff040fc93840: pid 11 "idle: cpu2" spin locks held: cpuid = 3 curthread = 0xffffff040fc93580: pid 10 "idle: cpu3" curpcb = 0xffffffffac060d40 fpcurthread = none idlethread = 0xffffff040fc93580: pid 10 "idle: cpu3" spin locks held: db> show locks db> show alllocks Process 53462 (sshd) thread 0xffffff008ba852c0 (100238) exclusive sx so_rcv_sx r = 0 (0xffffff03b3932928) locked _at_ /usr/src/sys/kern/uipc_sockbuf.c:141 Process 37123 (sshd) thread 0xffffff03084d5580 (100259) exclusive sx so_rcv_sx r = 0 (0xffffff034d8a0be0) locked _at_ /usr/src/sys/kern/uipc_sockbuf.c:141 Process 76971 (sshd) thread 0xffffff034db08b00 (100123) exclusive sx so_rcv_sx r = 0 (0xffffff02e3cc4be0) locked _at_ /usr/src/sys/kern/uipc_sockbuf.c:141 Process 827 (sshd) thread 0xffffff034da09840 (100119) exclusive sx so_rcv_sx r = 0 (0xffffff0349f21670) locked _at_ /usr/src/sys/kern/uipc_sockbuf.c:141 Process 18 (swi2: cambio) thread 0xffffff040fc5b840 (100015) exclusive sleep mutex Giant r = 0 (0xffffffff80608820) locked _at_ /usr/src/sys/cam/cam_xpt.c:7107 db> show lockedvnods Locked vnodes db> alltrace Tracing command perl5.8.8 pid 66384 tid 100138 td 0xffffff0345b6f2c0 kdb_enter() at kdb_enter+0x2f assert_vop_locked() at assert_vop_locked+0x77 VOP_READ_APV() at VOP_READ_APV+0x8a vn_read() at vn_read+0x244 dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800c7aa2c, rsp = 0x7fffffffc758, rbp = 0 --- Tracing command httpd pid 66383 tid 100199 td 0xffffff030581f000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee lf_advlock() at lf_advlock+0x29c VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x9b flock() at flock+0x125 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (131, FreeBSD ELF64, flock), rip = 0x800fb6e4c, rsp = 0x7fffffffeb38, rbp = 0x804cb1028 --- Tracing command httpd pid 66331 tid 100171 td 0xffffff02e2c00000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 poll() at poll+0x538 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (209, FreeBSD ELF64, poll), rip = 0x800fd006c, rsp = 0x7fffffff82c8, rbp = 0x801219400 --- Tracing command httpd pid 66292 tid 100213 td 0xffffff02e2c002c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _sleep() at _sleep+0x34d kern_kevent() at kern_kevent+0x2e9 kevent() at kevent+0x90 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (363, FreeBSD ELF64, kevent), rip = 0x8010083cc, rsp = 0x7fffffff86e8, rbp = 0x7fffffff875c --- Tracing command httpd pid 66290 tid 100353 td 0xffffff02dd7fa840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 poll() at poll+0x538 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (209, FreeBSD ELF64, poll), rip = 0x800fd006c, rsp = 0x7fffffffe818, rbp = 0x801219400 --- Tracing command httpd pid 66288 tid 100056 td 0xffffff00663f8000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 poll() at poll+0x538 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (209, FreeBSD ELF64, poll), rip = 0x800fd006c, rsp = 0x7fffffffe818, rbp = 0x801219400 --- Tracing command httpd pid 66279 tid 100125 td 0xffffff034db08580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 poll() at poll+0x538 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (209, FreeBSD ELF64, poll), rip = 0x800fd006c, rsp = 0x7fffffffe818, rbp = 0x801219400 --- Tracing command httpd pid 66277 tid 100321 td 0xffffff037e09e2c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 poll() at poll+0x538 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (209, FreeBSD ELF64, poll), rip = 0x800fd006c, rsp = 0x7fffffffe818, rbp = 0x801219400 --- Tracing command httpd pid 66275 tid 100287 td 0xffffff02ff4982c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 poll() at poll+0x538 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (209, FreeBSD ELF64, poll), rip = 0x800fd006c, rsp = 0x7fffffffe818, rbp = 0x801219400 --- Tracing command httpd pid 66274 tid 100166 td 0xffffff032cd14000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_kevent() at kern_kevent+0x2e9 kevent() at kevent+0x90 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (363, FreeBSD ELF64, kevent), rip = 0x8010083cc, rsp = 0x7fffffffeb08, rbp = 0x7fffffffeb74 --- Tracing command httpd pid 66267 tid 100290 td 0xffffff03b6a46840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 poll() at poll+0x538 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (209, FreeBSD ELF64, poll), rip = 0x800fd006c, rsp = 0x7fffffffe8a8, rbp = 0x801219400 --- Tracing command httpd pid 66210 tid 100231 td 0xffffff033ff762c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee lf_advlock() at lf_advlock+0x29c VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0x9b flock() at flock+0x125 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (131, FreeBSD ELF64, flock), rip = 0x800fb6e4c, rsp = 0x7fffffffeb38, rbp = 0x804cb1028 --- Tracing command httpd pid 66206 tid 100379 td 0xffffff022511fb00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 poll() at poll+0x538 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (209, FreeBSD ELF64, poll), rip = 0x800fd006c, rsp = 0x7fffffffe818, rbp = 0x801219400 --- Tracing command httpd pid 66203 tid 100205 td 0xffffff030939db00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 poll() at poll+0x538 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (209, FreeBSD ELF64, poll), rip = 0x800fd006c, rsp = 0x7fffffffe818, rbp = 0x801219400 --- Tracing command httpd pid 66031 tid 100149 td 0xffffff034600d000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 poll() at poll+0x538 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (209, FreeBSD ELF64, poll), rip = 0x800fd006c, rsp = 0x7fffffffe8a8, rbp = 0x801219400 --- Tracing command sftp-server pid 53467 tid 100078 td 0xffffff00663f8b00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _cv_wait_sig() at _cv_wait_sig+0x158 kern_select() at kern_select+0x80f select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x800d0a9ac, rsp = 0x7fffffffab68, rbp = 0x80151e100 --- Tracing command csh pid 53465 tid 100359 td 0xffffff030e579580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_sigsuspend() at kern_sigsuspend+0xa0 sigsuspend() at sigsuspend+0x40 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (341, FreeBSD ELF64, sigsuspend), rip = 0x80094a94c, rsp = 0x7fffffffe5f8, rbp = 0x5b7c08 --- Tracing command sshd pid 53464 tid 100192 td 0xffffff02ff080580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _cv_wait_sig() at _cv_wait_sig+0x158 kern_select() at kern_select+0x80f select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x8016e79ac, rsp = 0x7fffffffe008, rbp = 0x7fffffffe078 --- Tracing command sshd pid 53462 tid 100238 td 0xffffff008ba852c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee soreceive_generic() at soreceive_generic+0xf31 soo_read() at soo_read+0x37 dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x8016e7a2c, rsp = 0x7fffffffe008, rbp = 0x4 --- Tracing command ssh pid 3194 tid 100383 td 0xffffff022511f000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _cv_wait_sig() at _cv_wait_sig+0x158 kern_select() at kern_select+0x80f select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x800f349ac, rsp = 0x7fffffffbef8, rbp = 0x7fffffffbff8 --- Tracing command emacs pid 69351 tid 100122 td 0xffffff034da09000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 thread_suspend_check() at thread_suspend_check+0x12d userret() at userret+0x12c syscall() at syscall+0x2c6 Xfast_syscall() at Xfast_syscall+0xab --- syscall (0, FreeBSD ELF64, nosys), rip = 0x800baba3c, rsp = 0x7fffffffe398, rbp = 0x3 --- Tracing command ping pid 38702 tid 100241 td 0xffffff008bb04840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 kern_select() at kern_select+0x4fc select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x80093f9ac, rsp = 0x7ffffffee6d8, rbp = 0x7ffffffee880 --- Tracing command bash pid 37157 tid 100345 td 0xffffff032f666840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee ttysleep() at ttysleep+0x1b ttread() at ttread+0x312 ptsread() at ptsread+0x52 giant_read() at giant_read+0x51 devfs_read_f() at devfs_read_f+0x6c dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800bf4a2c, rsp = 0x7fffffffe6f8, rbp = 0 --- Tracing command su pid 37146 tid 100380 td 0xffffff022511f840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_wait() at kern_wait+0x39e wait4() at wait4+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009c8d8c, rsp = 0x7fffffffe6e8, rbp = 0x800b6f8a0 --- Tracing command bash pid 37142 tid 100237 td 0xffffff008ba85580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_wait() at kern_wait+0x39e wait4() at wait4+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (7, FreeBSD ELF64, wait4), rip = 0x800b7cd8c, rsp = 0x7fffffffe828, rbp = 0x800e315b0 --- Tracing command sshd pid 37141 tid 100401 td 0xffffff0296e13580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _cv_wait_sig() at _cv_wait_sig+0x158 kern_select() at kern_select+0x80f select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x8016e79ac, rsp = 0x7fffffffe008, rbp = 0x7fffffffe078 --- Tracing command sshd pid 37123 tid 100259 td 0xffffff03084d5580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee soreceive_generic() at soreceive_generic+0xf31 soo_read() at soo_read+0x37 dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x8016e7a2c, rsp = 0x7fffffffe008, rbp = 0x4 --- Tracing command emacs pid 77253 tid 100127 td 0xffffff034db08000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 thread_suspend_check() at thread_suspend_check+0x12d userret() at userret+0x12c syscall() at syscall+0x2c6 Xfast_syscall() at Xfast_syscall+0xab --- syscall (0, FreeBSD ELF64, nosys), rip = 0x800baba3c, rsp = 0x7fffffffe3f8, rbp = 0x3 --- Tracing command bash pid 77246 tid 100128 td 0xffffff034d8c7b00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_wait() at kern_wait+0x39e wait4() at wait4+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (7, FreeBSD ELF64, wait4), rip = 0x800b7cd8c, rsp = 0x7fffffffe788, rbp = 0x800e32560 --- Tracing command su pid 77235 tid 100049 td 0xffffff004ba462c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_wait() at kern_wait+0x39e wait4() at wait4+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009c8d8c, rsp = 0x7fffffffe6d8, rbp = 0x800b6f8a0 --- Tracing command bash pid 77054 tid 100275 td 0xffffff02b13e6580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_wait() at kern_wait+0x39e wait4() at wait4+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (7, FreeBSD ELF64, wait4), rip = 0x800b7cd8c, rsp = 0x7fffffffe7c8, rbp = 0x800e32110 --- Tracing command sh pid 77001 tid 100071 td 0xffffff007b99c2c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_wait() at kern_wait+0x39e wait4() at wait4+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (7, FreeBSD ELF64, wait4), rip = 0x800931d8c, rsp = 0x7fffffffe948, rbp = 0x12cc9 --- Tracing command sshd pid 77000 tid 100193 td 0xffffff02ff0802c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _cv_wait_sig() at _cv_wait_sig+0x158 kern_select() at kern_select+0x80f select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x8016e79ac, rsp = 0x7fffffffe008, rbp = 0x7fffffffe078 --- Tracing command sshd pid 76971 tid 100123 td 0xffffff034db08b00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee soreceive_generic() at soreceive_generic+0xf31 soo_read() at soo_read+0x37 dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x8016e7a2c, rsp = 0x7fffffffe008, rbp = 0x4 --- Tracing command bash pid 831 tid 100131 td 0xffffff034d8c72c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee ttysleep() at ttysleep+0x1b ttread() at ttread+0x312 ptsread() at ptsread+0x52 giant_read() at giant_read+0x51 devfs_read_f() at devfs_read_f+0x6c dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800bf4a2c, rsp = 0x7fffffffe778, rbp = 0 --- Tracing command sshd pid 830 tid 100129 td 0xffffff034d8c7840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _cv_wait_sig() at _cv_wait_sig+0x158 kern_select() at kern_select+0x80f select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x8016e79ac, rsp = 0x7fffffffe008, rbp = 0x7fffffffe078 --- Tracing command sshd pid 827 tid 100119 td 0xffffff034da09840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee soreceive_generic() at soreceive_generic+0xf31 soo_read() at soo_read+0x37 dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x8016e7a2c, rsp = 0x7fffffffe008, rbp = 0x4 --- Tracing command getty pid 765 tid 100105 td 0xffffff027a490580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee ttysleep() at ttysleep+0x1b ttread() at ttread+0x312 giant_read() at giant_read+0x51 devfs_read_f() at devfs_read_f+0x6c dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800829a2c, rsp = 0x7fffffffeda8, rbp = 0x5088c0 --- Tracing command getty pid 764 tid 100055 td 0xffffff004ba65000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee ttysleep() at ttysleep+0x1b ttread() at ttread+0x312 giant_read() at giant_read+0x51 devfs_read_f() at devfs_read_f+0x6c dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800829a2c, rsp = 0x7fffffffeda8, rbp = 0x5088c0 --- Tracing command getty pid 763 tid 100116 td 0xffffff010b1482c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee ttysleep() at ttysleep+0x1b ttread() at ttread+0x312 giant_read() at giant_read+0x51 devfs_read_f() at devfs_read_f+0x6c dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800829a2c, rsp = 0x7fffffffeda8, rbp = 0x5088c0 --- Tracing command getty pid 762 tid 100114 td 0xffffff010b148840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee ttysleep() at ttysleep+0x1b ttread() at ttread+0x312 giant_read() at giant_read+0x51 devfs_read_f() at devfs_read_f+0x6c dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800829a2c, rsp = 0x7fffffffeda8, rbp = 0x5088c0 --- Tracing command getty pid 761 tid 100112 td 0xffffff027a5ef000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee ttysleep() at ttysleep+0x1b ttread() at ttread+0x312 giant_read() at giant_read+0x51 devfs_read_f() at devfs_read_f+0x6c dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800829a2c, rsp = 0x7fffffffeda8, rbp = 0x5088c0 --- Tracing command getty pid 760 tid 100111 td 0xffffff027a5ef2c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee ttysleep() at ttysleep+0x1b ttread() at ttread+0x312 giant_read() at giant_read+0x51 devfs_read_f() at devfs_read_f+0x6c dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800829a2c, rsp = 0x7fffffffeda8, rbp = 0x5088c0 --- Tracing command getty pid 759 tid 100110 td 0xffffff027a5ef580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee ttysleep() at ttysleep+0x1b ttread() at ttread+0x312 giant_read() at giant_read+0x51 devfs_read_f() at devfs_read_f+0x6c dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800829a2c, rsp = 0x7fffffffeda8, rbp = 0x5088c0 --- Tracing command getty pid 758 tid 100108 td 0xffffff027a5efb00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee ttysleep() at ttysleep+0x1b ttread() at ttread+0x312 giant_read() at giant_read+0x51 devfs_read_f() at devfs_read_f+0x6c dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800829a2c, rsp = 0x7fffffffeda8, rbp = 0x5088c0 --- Tracing command getty pid 757 tid 100106 td 0xffffff027a4902c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee ttysleep() at ttysleep+0x1b ttread() at ttread+0x312 giant_read() at giant_read+0x51 devfs_read_f() at devfs_read_f+0x6c dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800829a2c, rsp = 0x7fffffffeda8, rbp = 0x5088c0 --- Tracing command inetd pid 742 tid 100067 td 0xffffff004ba46b00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _cv_wait_sig() at _cv_wait_sig+0x158 kern_select() at kern_select+0x80f select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x800a3e9ac, rsp = 0x7fffffffdae8, rbp = 0x800c39080 --- Tracing command cron pid 715 tid 100115 td 0xffffff010b148580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _sleep() at _sleep+0x34d kern_nanosleep() at kern_nanosleep+0x130 nanosleep() at nanosleep+0x59 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80081121c, rsp = 0x7fffffffec28, rbp = 0x46657ea8 --- Tracing command sendmail pid 709 tid 100083 td 0xffffff008bb26840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_sigsuspend() at kern_sigsuspend+0xa0 sigsuspend() at sigsuspend+0x40 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (341, FreeBSD ELF64, sigsuspend), rip = 0x800d0294c, rsp = 0x7fffffffcef8, rbp = 0 --- Tracing command sendmail pid 705 tid 100061 td 0xffffff00663b7000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 kern_select() at kern_select+0x4fc select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x800d8f9ac, rsp = 0x7fffffffc298, rbp = 0x3 --- Tracing command sshd pid 699 tid 100086 td 0xffffff010b027b00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _cv_wait_sig() at _cv_wait_sig+0x158 kern_select() at kern_select+0x80f select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x8016e79ac, rsp = 0x7fffffffe0e8, rbp = 0x9 --- Tracing command httpd pid 687 tid 100036 td 0xffffff040fc74840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _sleep() at _sleep+0x34d kern_nanosleep() at kern_nanosleep+0x130 nanosleep() at nanosleep+0x59 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80100721c, rsp = 0x7fffffffeae8, rbp = 0x801219400 --- Tracing command mysqld pid 675 tid 100283 td 0xffffff027f70bb00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee do_cv_wait() at do_cv_wait+0x5c7 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x8012a2e0c, rsp = 0x7ffffeaf2d88, rbp = 0 --- Tracing command mysqld pid 675 tid 100282 td 0xffffff040fbfd2c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee do_cv_wait() at do_cv_wait+0x5c7 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x8012a2e0c, rsp = 0x7ffffeb33d88, rbp = 0 --- Tracing command mysqld pid 675 tid 100281 td 0xffffff033ff76000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee do_cv_wait() at do_cv_wait+0x5c7 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x8012a2e0c, rsp = 0x7ffffeb74d88, rbp = 0 --- Tracing command mysqld pid 675 tid 100264 td 0xffffff02ffe94580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee do_cv_wait() at do_cv_wait+0x5c7 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x8012a2e0c, rsp = 0x7ffffebb5d88, rbp = 0 --- Tracing command mysqld pid 675 tid 100103 td 0xffffff027a490b00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_sigtimedwait() at kern_sigtimedwait+0x1fc sigwait() at sigwait+0x74 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (429, FreeBSD ELF64, sigwait), rip = 0x8012a312c, rsp = 0x7ffffebf6f28, rbp = 0x801501e10 --- Tracing command mysqld pid 675 tid 100100 td 0xffffff027ab31000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee do_cv_wait() at do_cv_wait+0x5c7 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x8012a2e0c, rsp = 0x7ffffedf7e98, rbp = 0 --- Tracing command mysqld pid 675 tid 100102 td 0xffffff040fbfd580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 kern_select() at kern_select+0x4fc select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x8013309ac, rsp = 0x7ffffeff8f08, rbp = 0 --- Tracing command mysqld pid 675 tid 100101 td 0xffffff040fbfd840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait_sig() at sleepq_timedwait_sig+0x2b _cv_timedwait_sig() at _cv_timedwait_sig+0x166 kern_select() at kern_select+0x4fc select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x8013309ac, rsp = 0x7fffff1f9f38, rbp = 0 --- Tracing command mysqld pid 675 tid 100099 td 0xffffff027ab312c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee do_cv_wait() at do_cv_wait+0x5c7 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x8012a2e0c, rsp = 0x7fffff5fbbf8, rbp = 0 --- Tracing command mysqld pid 675 tid 100098 td 0xffffff040f9d9000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee do_cv_wait() at do_cv_wait+0x5c7 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x8012a2e0c, rsp = 0x7fffff7fcbf8, rbp = 0 --- Tracing command mysqld pid 675 tid 100097 td 0xffffff040f9d92c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee do_cv_wait() at do_cv_wait+0x5c7 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x8012a2e0c, rsp = 0x7fffff9fdbf8, rbp = 0 --- Tracing command mysqld pid 675 tid 100096 td 0xffffff040f9d9580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee do_cv_wait() at do_cv_wait+0x5c7 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x8012a2e0c, rsp = 0x7fffffbfebf8, rbp = 0 --- Tracing command mysqld pid 675 tid 100091 td 0xffffff010afe6b00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _cv_wait_sig() at _cv_wait_sig+0x158 kern_select() at kern_select+0x80f select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x8013309ac, rsp = 0x7fffffffe518, rbp = 0xf --- Tracing command sh pid 644 tid 100092 td 0xffffff010afe6840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_wait() at kern_wait+0x39e wait4() at wait4+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (7, FreeBSD ELF64, wait4), rip = 0x800931d8c, rsp = 0x7fffffffe228, rbp = 0x283 --- Tracing command syslogd pid 536 tid 100090 td 0xffffff010b027000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _cv_wait_sig() at _cv_wait_sig+0x158 kern_select() at kern_select+0x80f select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x80082c9ac, rsp = 0x7fffffffd988, rbp = 0x800a1a0f0 --- Tracing command devd pid 469 tid 100060 td 0xffffff00663b72c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _cv_wait_sig() at _cv_wait_sig+0x158 kern_select() at kern_select+0x80f select() at select+0x3e syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (93, FreeBSD ELF64, select), rip = 0x437ebc, rsp = 0x7fffffffe938, rbp = 0x7fffffffe960 --- Tracing command adjkerntz pid 110 tid 100066 td 0xffffff00663d6000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_sigsuspend() at kern_sigsuspend+0xa0 sigsuspend() at sigsuspend+0x40 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (341, FreeBSD ELF64, sigsuspend), rip = 0x80068a94c, rsp = 0x7fffffffed18, rbp = 0x6270 --- Tracing command softdepflush pid 38 tid 100042 td 0xffffff040f97b840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait() at sleepq_timedwait+0x40 _sleep() at _sleep+0x3b3 softdep_flush() at softdep_flush+0x330 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac1fad30, rbp = 0 --- Tracing command syncer pid 37 tid 100043 td 0xffffff040f97b580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait() at sleepq_wait+0x40 _sleep() at _sleep+0x21a sched_sync() at sched_sync+0x557 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac1f5d30, rbp = 0 --- Tracing command vnlru pid 36 tid 100044 td 0xffffff040f97b2c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait() at sleepq_timedwait+0x40 _sleep() at _sleep+0x3b3 vnlru_proc() at vnlru_proc+0x6de fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac1f0d30, rbp = 0 --- Tracing command bufdaemon pid 35 tid 100026 td 0xffffff040fc7db00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait() at sleepq_timedwait+0x40 _sleep() at _sleep+0x3b3 buf_daemon() at buf_daemon+0x136 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0d0d30, rbp = 0 --- Tracing command pagezero pid 34 tid 100027 td 0xffffff040fc7d840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait() at sleepq_timedwait+0x40 _sleep() at _sleep+0x3b3 vm_pagezero() at vm_pagezero+0x74 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0cbd30, rbp = 0 --- Tracing command idlepoll pid 33 tid 100028 td 0xffffff040fc7d580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait() at sleepq_timedwait+0x40 _sleep() at _sleep+0x3b3 poll_idle() at poll_idle+0x93 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0c6d30, rbp = 0 --- Tracing command vmdaemon pid 32 tid 100029 td 0xffffff040fc7d2c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait() at sleepq_wait+0x40 _sleep() at _sleep+0x21a vm_daemon() at vm_daemon+0x48 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0c1d30, rbp = 0 --- Tracing command pagedaemon pid 31 tid 100030 td 0xffffff040fa75b00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait() at sleepq_timedwait+0x40 _sleep() at _sleep+0x3b3 vm_pageout() at vm_pageout+0x28a fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac19ad30, rbp = 0 --- Tracing command swi0: sio pid 30 tid 100031 td 0xffffff040fa75840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 ithread_loop() at ithread_loop+0x340 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac195d30, rbp = 0 --- Tracing command irq1: atkbd0 pid 29 tid 100032 td 0xffffff040fa75580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 ithread_loop() at ithread_loop+0x340 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac190d30, rbp = 0 --- Tracing command irq15: ata1 pid 28 tid 100033 td 0xffffff040fa752c0 fork_trampoline() at fork_trampoline Tracing command irq14: ata0 pid 27 tid 100034 td 0xffffff040fa75000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 ithread_loop() at ithread_loop+0x340 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac186d30, rbp = 0 --- Tracing command irq16: arcmsr0 pid 26 tid 100018 td 0xffffff040fc5b000 cpustop_handler() at cpustop_handler+0x35 ipi_nmi_handler() at ipi_nmi_handler+0x2e trap() at trap+0x365 nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff8041ab11, rsp = 0xffffffffab59eff0, rbp = 0xffffffffac0a37d0 --- siocnclose() at siocnclose+0x21 sio_cnputc() at sio_cnputc+0x89 cnputc() at cnputc+0x6a putchar() at putchar+0x5f kvprintf() at kvprintf+0xd45 printf() at printf+0xe1 panic() at panic+0x145 xpt_done() at xpt_done+0x14a arcmsr_interrupt() at arcmsr_interrupt+0x2df ithread_loop() at ithread_loop+0x108 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0a3d30, rbp = 0 --- Tracing command irq19: em1 pid 25 tid 100019 td 0xffffff040fc7ab00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 ithread_loop() at ithread_loop+0x340 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac09cd30, rbp = 0 --- Tracing command irq18: em0 pid 24 tid 100020 td 0xffffff040fc7a840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 ithread_loop() at ithread_loop+0x340 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac097d30, rbp = 0 --- Tracing command irq9: acpi0 pid 23 tid 100021 td 0xffffff040fc71b00 fork_trampoline() at fork_trampoline Tracing command acpi_task_2 pid 22 tid 100022 td 0xffffff040fc71840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait() at sleepq_wait+0x40 _sleep() at _sleep+0x21a taskqueue_thread_loop() at taskqueue_thread_loop+0x71 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0e4d30, rbp = 0 --- Tracing command acpi_task_1 pid 9 tid 100023 td 0xffffff040fc71580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait() at sleepq_wait+0x40 _sleep() at _sleep+0x21a taskqueue_thread_loop() at taskqueue_thread_loop+0x71 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0dfd30, rbp = 0 --- Tracing command acpi_task_0 pid 8 tid 100024 td 0xffffff040fc712c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait() at sleepq_wait+0x40 _sleep() at _sleep+0x21a taskqueue_thread_loop() at taskqueue_thread_loop+0x71 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0dad30, rbp = 0 --- Tracing command kqueue taskq pid 7 tid 100025 td 0xffffff040fc71000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait() at sleepq_wait+0x40 _sleep() at _sleep+0x21a taskqueue_thread_loop() at taskqueue_thread_loop+0x71 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0d5d30, rbp = 0 --- Tracing command swi6: task queue pid 21 tid 100011 td 0xffffff040fc56580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 ithread_loop() at ithread_loop+0x340 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac079d30, rbp = 0 --- Tracing command swi6: Giant taskq pid 20 tid 100012 td 0xffffff040fc562c0 fork_trampoline() at fork_trampoline Tracing command thread taskq pid 6 tid 100013 td 0xffffff040fc7d000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait() at sleepq_wait+0x40 _sleep() at _sleep+0x21a taskqueue_thread_loop() at taskqueue_thread_loop+0x71 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0bcd30, rbp = 0 --- Tracing command swi5: + pid 19 tid 100014 td 0xffffff040fc5bb00 fork_trampoline() at fork_trampoline Tracing command swi2: cambio pid 18 tid 100015 td 0xffffff040fc5b840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sched_add() at sched_add+0x6d7 intr_event_schedule_thread() at intr_event_schedule_thread+0xbc intr_execute_handlers() at intr_execute_handlers+0x182 Xapic_isr1() at Xapic_isr1+0x7f --- interrupt, rip = 0xffffffff80156c77, rsp = 0xffffffffac0b2ad0, rbp = 0xffffffffac0b2bd0 --- camisr_runqueue() at camisr_runqueue+0x257 camisr() at camisr+0xaf ithread_loop() at ithread_loop+0x108 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0b2d30, rbp = 0 --- Tracing command xpt_thrd pid 5 tid 100016 td 0xffffff040fc5b580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait() at sleepq_wait+0x40 _sleep() at _sleep+0x21a xpt_scanner_thread() at xpt_scanner_thread+0x38 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0add30, rbp = 0 --- Tracing command yarrow pid 17 tid 100017 td 0xffffff040fc5b2c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait() at sleepq_timedwait+0x40 _sleep() at _sleep+0x3b3 random_kthread() at random_kthread+0x100 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac0a8d30, rbp = 0 --- Tracing command g_down pid 4 tid 100005 td 0xffffff040fc93000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait() at sleepq_timedwait+0x40 _sleep() at _sleep+0x3b3 g_io_schedule_down() at g_io_schedule_down+0x213 g_down_procbody() at g_down_procbody+0x7a fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac056d30, rbp = 0 --- Tracing command g_up pid 3 tid 100006 td 0xffffff040fc7a580 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait() at sleepq_timedwait+0x40 _sleep() at _sleep+0x3b3 g_io_schedule_up() at g_io_schedule_up+0x147 g_up_procbody() at g_up_procbody+0x7a fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac092d30, rbp = 0 --- Tracing command g_event pid 2 tid 100007 td 0xffffff040fc7a2c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_timedwait() at sleepq_timedwait+0x40 _sleep() at _sleep+0x3b3 g_event_procbody() at g_event_procbody+0xaf fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac08dd30, rbp = 0 --- Tracing command swi1: net pid 16 tid 100008 td 0xffffff040fc7a000 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 ithread_loop() at ithread_loop+0x340 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac088d30, rbp = 0 --- Tracing command swi3: vm pid 15 tid 100009 td 0xffffff040fc56b00 fork_trampoline() at fork_trampoline Tracing command swi4: clock sio pid 14 tid 100010 td 0xffffff040fc56840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 ithread_loop() at ithread_loop+0x340 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac07ed30, rbp = 0 --- Tracing command idle: cpu0 pid 13 tid 100000 td 0xffffff040fc56000 cpustop_handler() at cpustop_handler+0x35 ipi_nmi_handler() at ipi_nmi_handler+0x2e trap() at trap+0x365 nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff80284240, rsp = 0xffffffff806c56d0, rbp = 0xffffffffac06fc20 --- sched_runnable() at sched_runnable+0x20 cpu_idle() at cpu_idle+0x26 sched_idletd() at sched_idletd+0x2f fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac06fd30, rbp = 0 --- Tracing command idle: cpu1 pid 12 tid 100001 td 0xffffff040fc93b00 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 ipi_bitmap_handler() at ipi_bitmap_handler+0x91 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x76 --- interrupt, rip = 0xffffffff803da5d6, rsp = 0xffffffffac06abe0, rbp = 0xffffffffac06abf0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1a0 sched_idletd() at sched_idletd+0x34 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac06ad30, rbp = 0 --- Tracing command idle: cpu2 pid 11 tid 100002 td 0xffffff040fc93840 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sched_add() at sched_add+0x6d7 intr_event_schedule_thread() at intr_event_schedule_thread+0xbc intr_execute_handlers() at intr_execute_handlers+0x182 Xapic_isr1() at Xapic_isr1+0x7f --- interrupt, rip = 0xffffffff803da5d6, rsp = 0xffffffffac065be0, rbp = 0xffffffffac065bf0 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1a0 sched_idletd() at sched_idletd+0x34 fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac065d30, rbp = 0 --- Tracing command idle: cpu3 pid 10 tid 100003 td 0xffffff040fc93580 cpustop_handler() at cpustop_handler+0x35 ipi_nmi_handler() at ipi_nmi_handler+0x2e trap() at trap+0x365 nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff803e878c, rsp = 0xffffffffab5a3ff0, rbp = 0xffffffffac060c40 --- cpu_idle() at cpu_idle+0x2c fork_exit() at fork_exit+0xaa fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffffffac060d30, rbp = 0 --- Tracing command init pid 1 tid 100004 td 0xffffff040fc932c0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 sleepq_switch() at sleepq_switch+0xcb sleepq_wait_sig() at sleepq_wait_sig+0x28 _sleep() at _sleep+0x2ee kern_wait() at kern_wait+0x39e wait4() at wait4+0x38 syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (7, FreeBSD ELF64, wait4), rip = 0x40c4dc, rsp = 0x7fffffffe8a8, rbp = 0x60 --- Tracing command swapper pid 0 tid 0 td 0xffffffff80605fa0 sched_switch() at sched_switch+0xe1 mi_switch() at mi_switch+0x1c7 scheduler() at scheduler+0x2eb mi_startup() at mi_startup+0xd3 btext() at btext+0x2c db> call boot(0) KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x3a mi_switch() at mi_switch+0x2bf boot() at boot+0x3e db_fncall() at db_fncall+0xb4 db_command_loop() at db_command_loop+0x319 db_trap() at db_trap+0x63 kdb_trap() at kdb_trap+0xa8 trap() at trap+0x20a calltrap() at calltrap+0x8 --- trap 0x3, rip = 0xffffffff8028ffcf, rsp = 0xffffffffae76f910, rbp = 0xffffffffae76f920 --- kdb_enter() at kdb_enter+0x2f assert_vop_locked() at assert_vop_locked+0x77 VOP_READ_APV() at VOP_READ_APV+0x8a vn_read() at vn_read+0x244 dofileread() at dofileread+0x94 kern_readv() at kern_readv+0x60 read() at read+0x4a syscall() at syscall+0x20b Xfast_syscall() at Xfast_syscall+0xab --- syscall (3, FreeBSD ELF64, read), rip = 0x800c7aa2c, rsp = 0x7fffffffc758, rbp = 0 --- db> call cpu_reset(0) cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 1 Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-CURRENT #6: Tue May 22 14:07:46 PDT 2007 mreimer_at_gandalf.vpop.net:/usr/obj/usr/src/sys/GANDALF WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(R) CPU 5160 _at_ 3.00GHz (3000.02-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x6f6 Stepping = 6 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x4e3bd<SSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,<b15>,DCA> AMD Features=0x20100800<SYSCALL,NX,LM> AMD Features2=0x1<LAHF> Cores per package: 2 usable memory = 17170817024 (16375 MB) avail memory = 16629559296 (15859 MB) ACPI APIC Table: <PTLTD APIC > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 6 cpu3 (AP): APIC ID: 7 ioapic0 <Version 2.0> irqs 0-23 on motherboard ioapic1 <Version 2.0> irqs 24-47 on motherboard kbd1 at kbdmux0 acpi0: <PTLTD RSDT> on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: <ACPI CPU> on acpi0 acpi_throttle0: <ACPI CPU Throttling> on cpu0 cpu1: <ACPI CPU> on acpi0 acpi_throttle1: <ACPI CPU Throttling> on cpu1 acpi_throttle1: failed to attach P_CNT device_attach: acpi_throttle1 attach returned 6 cpu2: <ACPI CPU> on acpi0 acpi_throttle2: <ACPI CPU Throttling> on cpu2 acpi_throttle2: failed to attach P_CNT device_attach: acpi_throttle2 attach returned 6 cpu3: <ACPI CPU> on acpi0 acpi_throttle3: <ACPI CPU Throttling> on cpu3 acpi_throttle3: failed to attach P_CNT device_attach: acpi_throttle3 attach returned 6 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0 pci1: <ACPI PCI bus> on pcib1 pcib2: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci1 pci2: <ACPI PCI bus> on pcib2 pcib3: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci2 pci3: <ACPI PCI bus> on pcib3 pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3 pci4: <ACPI PCI bus> on pcib4 pcib5: <ACPI PCI-PCI bridge> at device 0.2 on pci3 pci5: <ACPI PCI bus> on pcib5 pcib6: <ACPI PCI-PCI bridge> irq 18 at device 2.0 on pci2 pci6: <ACPI PCI bus> on pcib6 em0: <Intel(R) PRO/1000 Network Connection Version - 6.5.0> port 0x2000-0x201f mem 0xd8400000-0xd841ffff irq 18 at device 0.0 on pci6 em0: Ethernet address: 00:30:48:33:96:a4 em0: [ITHREAD] em1: <Intel(R) PRO/1000 Network Connection Version - 6.5.0> port 0x2020-0x203f mem 0xd8420000-0xd843ffff irq 19 at device 0.1 on pci6 em1: Ethernet address: 00:30:48:33:96:a5 em1: [ITHREAD] pcib7: <ACPI PCI-PCI bridge> at device 0.3 on pci1 pci7: <ACPI PCI bus> on pcib7 pcib8: <ACPI PCI-PCI bridge> at device 4.0 on pci0 pci8: <ACPI PCI bus> on pcib8 pcib9: <ACPI PCI-PCI bridge> at device 6.0 on pci0 pci9: <ACPI PCI bus> on pcib9 pcib10: <PCI-PCI bridge> at device 0.0 on pci9 pci10: <PCI bus> on pcib10 arcmsr0: <Areca SATA Host Adapter RAID Controller (RAID6 capable) > mem 0xd8700000-0xd8700fff,0xd8000000-0xd83fffff irq 16 at device 14.0 on pci10 ARECA RAID ADAPTER0: Driver Version 1.20.00.14 2007-2-05 ARECA RAID ADAPTER0: FIRMWARE VERSION V1.42 2007-1-15 arcmsr0: [ITHREAD] pcib11: <PCI-PCI bridge> at device 0.2 on pci9 pci11: <PCI bus> on pcib11 pci0: <base peripheral> at device 8.0 (no driver attached) pcib12: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0 pci12: <ACPI PCI bus> on pcib12 pci0: <serial bus, USB> at device 29.0 (no driver attached) pci0: <serial bus, USB> at device 29.1 (no driver attached) pci0: <serial bus, USB> at device 29.2 (no driver attached) pci0: <serial bus, USB> at device 29.7 (no driver attached) pcib13: <ACPI PCI-PCI bridge> at device 30.0 on pci0 pci13: <ACPI PCI bus> on pcib13 vgapci0: <VGA-compatible display> port 0x3000-0x30ff mem 0xd0000000-0xd7ffffff,0xd8600000-0xd860ffff irq 18 at device 1.0 on pci13 isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel 63XXESB2 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0 ata0: <ATA channel 0> on atapci0 ata0: [ITHREAD] ata1: <ATA channel 1> on atapci0 ata1: [ITHREAD] pci0: <serial bus, SMBus> at device 31.3 (no driver attached) acpi_button0: <Power Button> on acpi0 atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A, console sio0: [FILTER] sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A sio1: [FILTER] fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 orm0: <ISA Option ROMs> at iomem 0xc0000-0xcafff,0xcb000-0xcbfff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 1.000 msec Waiting 5 seconds for SCSI devices to settle acd0: DMA limited to UDMA33, controller found non-ATA66 cable acd0: DVDROM <MATSHITADVD-ROM SR-8178/PZ16> at ata0-slave UDMA33 acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00 GEOM_LABEL: Label for provider acd0 is iso9660/FreeBSD_Install. acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00 da0 at arcmsr0 bus 0 target 0 lun 0 da0: <Areca gandalf-FreeBSD6 R001> Fixed Direct Access SCSI-5 device da0: 166.666MB/s transfers (83.333MHz, offset 32, 16bit) da0: 19072MB (39060480 512 byte sectors: 255H 63S/T 2431C) da1 at arcmsr0 bus 0 target 0 lun 1 da1: <Areca gandalf-FreeBSD7 R001> Fixed Direct Access SCSI-5 device da1: 166.666MB/s transfers (83.333MHz, offset 32, 16bit) da1: 19072MB (39060480 512 byte sectors: 255H 63S/T 2431C) da2 at arcmsr0 bus 0 target 0 lun 2 da2: <Areca gandalf-data R001> Fixed Direct Access SCSI-5 device da2: 166.666MB/s transfers (83.333MHz, offset 32, 16bit) da2: 2761840MB (5656249344 512 byte sectors: 255H 63S/T 352085C) da3 at arcmsr0 bus 0 target 0 lun 3 da3: <Areca gandalf-swap R001> Fixed Direct Access SCSI-5 device da3: 166.666MB/s transfers (83.333MHz, offset 32, 16bit) da3: 61032MB (124993536 512 byte sectors: 255H 63S/T 7780C) SMP: AP CPU #1 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #2 Launched! Trying to mount root from ufs:/dev/da1s1a WARNING: reducing size to maximum of 67108864 blocks per swap unit ipfw2 (+ipv6) initialized, divert loadable, rule-based forwarding disabled, default to deny, logging disabled em0: link state changed to UPReceived on Tue Jun 05 2007 - 19:52:11 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:11 UTC