This was during the single- to multi-user transition on boot; I was going to skip the output from the LORs, but I suspect that some of them might be relevant; see below. I can leave the system in this state for a whle, but I normally power it off (to reduce heat, noise, and electricity consumption in a room my spouse uses). Please note that this is using a GENERIC kernel. Here's a cut/paste from the serial console: GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #144 r207433: Fri Apr 30 07:13:53 PDT 2010 root_at_freebeast.catwhisker.org:/usr/obj/usr/src/sys/GENERIC i386 WARNING: WITNESS option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 3.60GHz (3614.55-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf41 Family = f Model = 4 Stepping = 1 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x659d<SSE3,DTES64,MON,DS_CPL,EST,TM2,CNXT-ID,CX16,xTPR> AMD Features=0x20100000<NX,LM> TSC: P-state invariant real memory = 2147483648 (2048 MB) avail memory = 2086129664 (1989 MB) ACPI APIC Table: <PTLTD APIC > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 2 package(s) x 1 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 6 ioapic0 <Version 2.0> irqs 0-23 on motherboard ioapic1 <Version 2.0> irqs 24-47 on motherboard ioapic2 <Version 2.0> irqs 48-71 on motherboard kbd1 at kbdmux0 acpi0: <PTLTD RSDT> on motherboard ... aacd0: <RAID 1 (Mirror)> on aac0 aacd0: 34970MB (71619584 sectors) aacd1: <RAID 1 (Mirror)> on aac0 aacd1: 69974MB (143307008 sectors) uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub4: 8 ports with 8 removable, self powered ses0 at aacp0 bus 0 scbus0 target 6 lun 0 ses0: <SUPER GEM318 0> Fixed Uninstalled SCSI-2 device ses0: 3.300MB/s transfers ses0: SAF-TE Compliant Device pass0 at aacp0 bus 0 scbus0 target 0 lun 0 pass0: <SEAGATE ST336754LC 0003> Fixed Uninstalled SCSI-3 device pass0: 3.300MB/s transfers pass1 at aacp0 bus 0 scbus0 target 1 lun 0 pass1: <SEAGATE ST336754LC 0003> Fixed Uninstalled SCSI-3 device pass1: 3.300MB/s transfers pass2 at aacp0 bus 0 scbus0 target 2 lun 0 pass2: <SEAGATE ST373454LC 0005> Fixed Uninstalled SCSI-3 device pass2: 3.300MB/s transfers pass3 at aacp0 bus 0 scbus0 target 3 lun 0 pass3: <SEAGATE ST373454LC 0005> Fixed Uninstalled SCSI-3 device pass3: 3.300MB/s transfers SMP: AP CPU #1 Launched! WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/aacd0s4a Setting hostuuid: 80f1e964-dc63-0010-89d5-0030482d326a. Setting hostid: 0xc74551dd. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/aacd0s4a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s4a: clean, 499913 free (1473 frags, 62305 blocks, 0.2% fragmentation) /dev/aacd0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd1s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s1a: clean, 424969 free (1913 frags, 52882 blocks, 0.3% fragmentation) /dev/aacd1s1e: clean, 4818032 free (263136 frags, 569362 blocks, 2.0% fragmentation) /dev/aacd0s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s1d: clean, 1031838 free (4190 frags, 128456 blocks, 0.2% fragmentation) /dev/aWARNING: TMPFS is considered to be a highly experimental feature in FreeBSD. acd1s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd1s1d: clean, 11736234 free (197738 frags, 1442312 blocks, 1.2% fragmentation) /dev/aacd0s2a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s2a: clean, 596278 free (1118 frags, 74395 blocks, 0.2% fragmentation) /dev/aacd0s2d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s2d: clean, 1109224 free (22320 frags, 135863 blocks, 1.2% fragmentation) /dev/aacd0s3a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s3a: clean, 578047 free (1031 frags, 72127 blocks, 0.1% fragmentation) /dev/aacd0s3d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s3d: clean, 1071557 free (38805 frags, 129094 blocks, 2.2% fragmentation) /dev/aacd0s4d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s4d: clean, 1078573 free (59909 frags, 127333 blocks, 3.3% fragmentation) /dev/aacd0s4f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s4f: clean, 2106253 free (349 frags, 263238 blocks, 0.0% fragmentation) Mounting local file systems:. Setting hostname: freebeast.catwhisker.org. lock order reversal: 1st 0xc64c85a8 ufs (ufs) _at_ /usr/src/sys/kern/vfs_lookup.c:502 2nd 0xd94ff1a0 bufwait (bufwait) _at_ /usr/src/sys/ufs/ffs/ffs_softdep.c:11362 3rd 0xc64e69e8 ufs (ufs) _at_ /usr/src/sys/kern/vfs_subr.c:2091 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebdd2300,c08e8615,c08d898b,c0cc700a,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08d898b,c0cc700a,c553d030,c5540568,ebdd235c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc700a,c64e69e8,c0cb932f,c5540568,c0cce101,...) at _witness_debugger+0x25 witness_checkorder(c64e69e8,9,c0cce101,82b,0,...) at witness_checkorder+0x839 __lockmgr_args(c64e69e8,80100,c64e6a08,0,0,...) at __lockmgr_args+0x7f9 ffs_lock(ebdd2480,c08e83bb,c0ccd5a6,80100,c64e6990,...) at ffs_lock+0x8a VOP_LOCK1_APV(c0dd1b20,ebdd2480,c5eee764,c0dec560,c64e6990,...) at VOP_LOCK1_APV+0xb5 _vn_lock(c64e6990,80100,c0cce101,82b,4,...) at _vn_lock+0x5e vget(c64e6990,80100,c5eee6c0,50,0,...) at vget+0xb9 vfs_hash_get(c64bf798,78c12,80000,c5eee6c0,ebdd25d0,...) at vfs_hash_get+0xe6 ffs_vgetf(c64bf798,78c12,80000,ebdd25d0,1,...) at ffs_vgetf+0x49 softdep_sync_metadata(c64c8550,0,c0cea2f8,147,0,...) at softdep_sync_metadata+0xc92 ffs_syncvnode(c64c8550,1,c5eee6c0,ebdd2690,246,...) at ffs_syncvnode+0x3e2 ffs_truncate(c64c8550,200,0,880,c5585380,...) at ffs_truncate+0x862 ufs_direnter(c64c8550,c64e6990,ebdd2944,ebdd2bd4,0,...) at ufs_direnter+0x8d4 ufs_makeinode(ebdd2bd4,0,ebdd2b30,ebdd2a8c,c0c002d5,...) at ufs_makeinode+0x557 ufs_create(ebdd2b30,ebdd2b48,0,0,ebdd2ba8,...) at ufs_create+0x30 VOP_CREATE_APV(c0dd1b20,ebdd2b30,ebdd2bd4,ebdd2ac8,0,...) at VOP_CREATE_APV+0xa5 vn_open_cred(ebdd2ba8,ebdd2c5c,1a4,0,c5585380,...) at vn_open_cred+0x215 vn_open(ebdd2ba8,ebdd2c5c,1a4,c5f54038,0,...) at vn_open+0x3b kern_openat(c5eee6c0,ffffff9c,804c5e8,0,602,...) at kern_openat+0x125 kern_open(c5eee6c0,804c5e8,0,601,21b6,...) at kern_open+0x35 open(c5eee6c0,ebdd2cf8,c0cfcac4,c0ca746c,c5f692a8,...) at open+0x30 syscall(ebdd2d38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (5, FreeBSD ELF32, open), eip = 0x281743c3, esp = 0xbfbfec4c, ebp = 0xbfbfecb8 --- Starting Network: lo0 em0 em1. lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=3<RXCSUM,TXCSUM> inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC> ether 00:30:48:2d:32:6a inet 172.16.8.10 netmask 0xffffff00 broadcast 172.16.8.255 inet6 fe80::230:48ff:fe2d:326a%em0 prefixlen 64 tentative scopeid 0x1 nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> media: Ethernet autoselect status: no carrier em1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC> ether 00:30:48:2d:32:6b media: Ethernet autoselect status: no carrier Starting devd. Starting Network: em1. em1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC> ether 00:30:48:2d:32:6b media: Ethernet autoselect status: no carrier add net default: gateway 172.16.8.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout Creating and/or trimming log files. Starting syslogd. No core dumps found. Starting rpcbind. NFS access cache time=60 Setting NIS domain: lmdhw.com. Startlock order reversal: ing ypbind. Sta 1st 0xc5f89990 vm object (standard object) _at_ /usr/src/sys/vm/vm_fault.c:1280 rting amd. 2nd 0xc0f9b180 page lock (page lock) _at_ /usr/src/sys/vm/vm_fault.c:1299 3rd 0xc668cbb0 vm object (standard object) _at_ /usr/src/sys/vm/vm_fault.c:1249 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bad0,c08e8615,c08d898b,c0cc700a,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08d898b,c0cc700a,c553a2b0,c553d648,ebe0bb2c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc700a,c668cbb0,c0cc720f,c553d648,c0cec528,...) at _witness_debugger+0x25 witness_checkorder(c668cbb0,9,c0cec528,4e1,0,...) at witness_checkorder+0x839 _mtx_lock_flags(c668cbb0,0,c0cec528,4e1,5,...) at _mtx_lock_flags+0xc4 vm_fault_copy_entry(c64f34b0,c5586c30,c5faae58,c5faaca8,ebe0bc48,...) at vm_fault_copy_entry+0x228 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 75 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c5faae58,c5faaca8,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 75 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,1,0,517,5,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c5faa1f8,c5f5fc18,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 77 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c5faa1f8,c5f5fc18,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 77 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,391,0,517,3,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c5f5f7e0,c5f50480,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 333 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c5f5f7e0,c5f50480,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 333 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,30,0,517,3,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c15a93a8,c5f50b88,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 429 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c15a93a8,c5f50b88,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 429 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,2,0,517,5,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c5faaa68,c5f50ca8,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 433 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c5faaa68,c5f50ca8,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 433 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,13,0,517,3,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c5f50cf0,c5faa360,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 471 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c5f50cf0,c5faa360,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 471 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,7,0,517,3,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c15a9558,c5faa0d8,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 485 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c15a9558,c5faa0d8,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 485 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,1,0,517,5,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c5fba8b8,c5f50900,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 487 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c5fba8b8,c5f50900,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 487 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,103,0,517,3,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c5fba2d0,c5f5fdc8,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 1005 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c5fba2d0,c5f5fdc8,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 1005 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,6,0,517,5,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c5f50630,c5faa9d8,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 1017 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c5f50630,c5faa9d8,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 1017 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,17,0,517,3,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c5f5f678,c5faae10,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 1063 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c5f5f678,c5faae10,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 1063 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,400,0,517,3,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c5f5fe10,c65b5090,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1063 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34 vm_map_entry_create(c64f34b0,c5586c30,c5f5fe10,c65b5090,ebe0bc48,...) at vm_map_entry_create+0x4d vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1063 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34 vm_object_allocate(0,20,0,517,3,...) at vm_object_allocate+0x2d vm_fault_copy_entry(c64f34b0,c5586c30,c65b5750,c5f50510,ebe0bc48,...) at vm_fault_copy_entry+0x62 vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- lock order reversal: 1st 0xc0f9a780 page lock (page lock) _at_ /usr/src/sys/vm/vm_fault.c:1299 2nd 0xc5f8d330 process lock (process lock) _at_ /usr/src/sys/vm/swap_pager.c:208 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb40,c08e8615,c08d898b,c0cc6ff1,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08d898b,c0cc6ff1,c553a2b0,c5539340,ebe0bb9c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc6ff1,c5f8d330,c0cbf526,c5539340,c0ceb66c,...) at _witness_debugger+0x25 witness_checkorder(c5f8d330,9,c0ceb66c,d0,0,...) at witness_checkorder+0x839 _mtx_lock_flags(c5f8d330,0,c0ceb66c,d0,c5586c30,...) at _mtx_lock_flags+0xc4 swap_reserve_by_uid(945000,0,c5567400,ebe0bc60,c0878318,...) at swap_reserve_by_uid+0x12e swap_reserve(945000,0,2,0,0,...) at swap_reserve+0x2b fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x298 fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- lock order reversal: (sleepable after non-sleepable) 1st 0xc0f9a780 page lock (page lock) _at_ /usr/src/sys/vm/vm_fault.c:1299 2nd 0xc0e214f0 proctree (proctree) _at_ /usr/src/sys/kern/kern_fork.c:329 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb7c,c08e8615,c08d898b,c0cc6ff1,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08d898b,c0cc6ff1,c553a2b0,c5539068,ebe0bbd8,...) at kdb_backtrace+0x29 _witness_debugger(c0cc6ff1,c0e214f0,c0cbfd3f,c5539068,c0cbbe2a,...) at _witness_debugger+0x25 witness_checkorder(c0e214f0,1,c0cbbe2a,149,0,...) at witness_checkorder+0x839 _sx_slock(c0e214f0,0,c0cbbe2a,149,0,...) at _sx_slock+0x85 fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x320 fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "4096" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1127 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb30,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb68,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb7c,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0c5929c,c0cbbe2a,...) at witness_warn+0x1fd uma_zalloc_arg(c15b0700,0,102,2,37d,...) at uma_zalloc_arg+0x34 malloc(abc,c0db7000,102,1869f,37d,...) at malloc+0xe4 sigacts_alloc(c5f8d880,0,c0cbbe2a,1de,0,...) at sigacts_alloc+0x23 fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x858 fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- uma_zalloc_arg: zone "256" with the following non-sleepable locks held: exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1127 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb00,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb38,...) at kdb_backtrace+0x29 _witness_debugger(c0cc659d,ebe0bb4c,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cec231,c0c56d98,109,...) at witness_warn+0x1fd uma_zalloc_arg(c15b1700,0,102,2,37d,...) at uma_zalloc_arg+0x34 malloc(b8,c0db2b84,102,c08923ea,37d,...) at malloc+0xe4 fdinit(c65aa200,ebe0bc00,c08a2e55,c6675aa8,c0cc0bde,...) at fdinit+0x28 fdcopy(c65aa200,0,c0cbbe2a,1de,0,...) at fdcopy+0x21 fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x893 fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- lock order reversal: (sleepable after non-sleepable) 1st 0xc0f9a780 page lock (page lock) _at_ /usr/src/sys/vm/vm_fault.c:1299 2nd 0xc65aa22c filedesc structure (filedesc structure) _at_ /usr/src/sys/kern/kern_descrip.c:1580 KDB: stack backtrace: db_trace_self_wrapper(c0cc4017,ebe0bb30,c08e8615,c08d898b,c0cc6ff1,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c08d898b,c0cc6ff1,c553a2b0,c553d308,ebe0bb8c,...) at kdb_backtrace+0x29 _witness_debugger(c0cc6ff1,c65aa22c,c0cbb557,c553d308,c0cbb30f,...) at _witness_debugger+0x25 witness_checkorder(c65aa22c,9,c0cbb30f,62c,0,...) at witness_checkorder+0x839 _sx_xlock(c65aa22c,0,c0cbb30f,62c,37d,...) at _sx_xlock+0x85 fdinit(c65aa200,ebe0bc00,c08a2e55,c6675aa8,c0cc0bde,...) at fdinit+0x6c fdcopy(c65aa200,0,c0cbbe2a,1de,0,...) at fdcopy+0x21 fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x893 fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29 syscall(ebe0bd38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- System call fork returning with the following locks held: exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1127 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 panic: witness_warn cpuid = 1 KDB: enter: panic [ thread pid 892 tid 100089 ] Stopped at kdb_enter+0x3a: movl $0,kdb_why db> bt Tracing pid 892 tid 100089 td 0xc5f92000 kdb_enter(c0cc0b06,c0cc0b06,c0c68e9c,ebe0bc54,1,...) at kdb_enter+0x3a panic(c0c68e9c,c0c434fd,0,0,0,...) at panic+0x136 witness_warn(2,0,c0cfcac4,c0cbbefd,c5f8d2a8,...) at witness_warn+0x1e9 syscall(ebe0bd38) at syscall+0x2d8 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (893), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 --- db> show locks exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1127 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299 db> Please let me know if additional information might be useful. I should be able to try various experiments & patches. As I type, I'm still building the kernel for CURRENT as of r207433 on my laptop; I should be able to report whether or not it exhibits similar behavior. Peace, david -- David H. Wolfskill david_at_catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key.
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:03 UTC