Panic _at_r207433: "System call fork returning with the following locks held"

From: David Wolfskill <david_at_catwhisker.org>
Date: Fri, 30 Apr 2010 09:19:53 -0700
This was during the single- to multi-user transition on boot; I was
going to skip the output from the LORs, but I suspect that some of
them might be relevant; see below.

I can leave the system in this state for a whle, but I normally power it
off (to reduce heat, noise, and electricity consumption in a room my
spouse uses).

Please note that this is using a GENERIC kernel.

Here's a cut/paste from the serial console:

GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2010 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-CURRENT #144 r207433: Fri Apr 30 07:13:53 PDT 2010
    root_at_freebeast.catwhisker.org:/usr/obj/usr/src/sys/GENERIC i386
WARNING: WITNESS option enabled, expect reduced performance.
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 3.60GHz (3614.55-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf41  Family = f  Model = 4  Stepping = 1
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x659d<SSE3,DTES64,MON,DS_CPL,EST,TM2,CNXT-ID,CX16,xTPR>
  AMD Features=0x20100000<NX,LM>
  TSC: P-state invariant
real memory  = 2147483648 (2048 MB)
avail memory = 2086129664 (1989 MB)
ACPI APIC Table: <PTLTD          APIC  >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 2 package(s) x 1 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  6
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic2 <Version 2.0> irqs 48-71 on motherboard
kbd1 at kbdmux0
acpi0: <PTLTD   RSDT> on motherboard

...

aacd0: <RAID 1 (Mirror)> on aac0
aacd0: 34970MB (71619584 sectors)
aacd1: <RAID 1 (Mirror)> on aac0
aacd1: 69974MB (143307008 sectors)
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
uhub4: 8 ports with 8 removable, self powered
ses0 at aacp0 bus 0 scbus0 target 6 lun 0
ses0: <SUPER GEM318 0> Fixed Uninstalled SCSI-2 device 
ses0: 3.300MB/s transfers
ses0: SAF-TE Compliant Device
pass0 at aacp0 bus 0 scbus0 target 0 lun 0
pass0: <SEAGATE ST336754LC 0003> Fixed Uninstalled SCSI-3 device 
pass0: 3.300MB/s transfers
pass1 at aacp0 bus 0 scbus0 target 1 lun 0
pass1: <SEAGATE ST336754LC 0003> Fixed Uninstalled SCSI-3 device 
pass1: 3.300MB/s transfers
pass2 at aacp0 bus 0 scbus0 target 2 lun 0
pass2: <SEAGATE ST373454LC 0005> Fixed Uninstalled SCSI-3 device 
pass2: 3.300MB/s transfers
pass3 at aacp0 bus 0 scbus0 target 3 lun 0
pass3: <SEAGATE ST373454LC 0005> Fixed Uninstalled SCSI-3 device 
pass3: 3.300MB/s transfers
SMP: AP CPU #1 Launched!
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/aacd0s4a
Setting hostuuid: 80f1e964-dc63-0010-89d5-0030482d326a.
Setting hostid: 0xc74551dd.
Entropy harvesting: interrupts ethernet point_to_point kickstart.
Starting file system checks:
/dev/aacd0s4a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s4a: clean, 499913 free (1473 frags, 62305 blocks, 0.2% fragmentation)
/dev/aacd0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd1s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s1a: clean, 424969 free (1913 frags, 52882 blocks, 0.3% fragmentation)
/dev/aacd1s1e: clean, 4818032 free (263136 frags, 569362 blocks, 2.0% fragmentation)
/dev/aacd0s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s1d: clean, 1031838 free (4190 frags, 128456 blocks, 0.2% fragmentation)
/dev/aWARNING: TMPFS is considered to be a highly experimental feature in FreeBSD.
acd1s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd1s1d: clean, 11736234 free (197738 frags, 1442312 blocks, 1.2% fragmentation)
/dev/aacd0s2a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s2a: clean, 596278 free (1118 frags, 74395 blocks, 0.2% fragmentation)
/dev/aacd0s2d: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s2d: clean, 1109224 free (22320 frags, 135863 blocks, 1.2% fragmentation)
/dev/aacd0s3a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s3a: clean, 578047 free (1031 frags, 72127 blocks, 0.1% fragmentation)
/dev/aacd0s3d: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s3d: clean, 1071557 free (38805 frags, 129094 blocks, 2.2% fragmentation)
/dev/aacd0s4d: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s4d: clean, 1078573 free (59909 frags, 127333 blocks, 3.3% fragmentation)
/dev/aacd0s4f: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/aacd0s4f: clean, 2106253 free (349 frags, 263238 blocks, 0.0% fragmentation)
Mounting local file systems:.
Setting hostname: freebeast.catwhisker.org.
lock order reversal:
 1st 0xc64c85a8 ufs (ufs) _at_ /usr/src/sys/kern/vfs_lookup.c:502
 2nd 0xd94ff1a0 bufwait (bufwait) _at_ /usr/src/sys/ufs/ffs/ffs_softdep.c:11362
 3rd 0xc64e69e8 ufs (ufs) _at_ /usr/src/sys/kern/vfs_subr.c:2091
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebdd2300,c08e8615,c08d898b,c0cc700a,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c08d898b,c0cc700a,c553d030,c5540568,ebdd235c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc700a,c64e69e8,c0cb932f,c5540568,c0cce101,...) at _witness_debugger+0x25
witness_checkorder(c64e69e8,9,c0cce101,82b,0,...) at witness_checkorder+0x839
__lockmgr_args(c64e69e8,80100,c64e6a08,0,0,...) at __lockmgr_args+0x7f9
ffs_lock(ebdd2480,c08e83bb,c0ccd5a6,80100,c64e6990,...) at ffs_lock+0x8a
VOP_LOCK1_APV(c0dd1b20,ebdd2480,c5eee764,c0dec560,c64e6990,...) at VOP_LOCK1_APV+0xb5
_vn_lock(c64e6990,80100,c0cce101,82b,4,...) at _vn_lock+0x5e
vget(c64e6990,80100,c5eee6c0,50,0,...) at vget+0xb9
vfs_hash_get(c64bf798,78c12,80000,c5eee6c0,ebdd25d0,...) at vfs_hash_get+0xe6
ffs_vgetf(c64bf798,78c12,80000,ebdd25d0,1,...) at ffs_vgetf+0x49
softdep_sync_metadata(c64c8550,0,c0cea2f8,147,0,...) at softdep_sync_metadata+0xc92
ffs_syncvnode(c64c8550,1,c5eee6c0,ebdd2690,246,...) at ffs_syncvnode+0x3e2
ffs_truncate(c64c8550,200,0,880,c5585380,...) at ffs_truncate+0x862
ufs_direnter(c64c8550,c64e6990,ebdd2944,ebdd2bd4,0,...) at ufs_direnter+0x8d4
ufs_makeinode(ebdd2bd4,0,ebdd2b30,ebdd2a8c,c0c002d5,...) at ufs_makeinode+0x557
ufs_create(ebdd2b30,ebdd2b48,0,0,ebdd2ba8,...) at ufs_create+0x30
VOP_CREATE_APV(c0dd1b20,ebdd2b30,ebdd2bd4,ebdd2ac8,0,...) at VOP_CREATE_APV+0xa5
vn_open_cred(ebdd2ba8,ebdd2c5c,1a4,0,c5585380,...) at vn_open_cred+0x215
vn_open(ebdd2ba8,ebdd2c5c,1a4,c5f54038,0,...) at vn_open+0x3b
kern_openat(c5eee6c0,ffffff9c,804c5e8,0,602,...) at kern_openat+0x125
kern_open(c5eee6c0,804c5e8,0,601,21b6,...) at kern_open+0x35
open(c5eee6c0,ebdd2cf8,c0cfcac4,c0ca746c,c5f692a8,...) at open+0x30
syscall(ebdd2d38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (5, FreeBSD ELF32, open), eip = 0x281743c3, esp = 0xbfbfec4c, ebp = 0xbfbfecb8 ---
Starting Network: lo0 em0 em1.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=3<RXCSUM,TXCSUM>
        inet 127.0.0.1 netmask 0xff000000 
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
        ether 00:30:48:2d:32:6a
        inet 172.16.8.10 netmask 0xffffff00 broadcast 172.16.8.255
        inet6 fe80::230:48ff:fe2d:326a%em0 prefixlen 64 tentative scopeid 0x1 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier
em1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
        ether 00:30:48:2d:32:6b
        media: Ethernet autoselect
        status: no carrier
Starting devd.
Starting Network: em1.
em1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=389b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC>
        ether 00:30:48:2d:32:6b
        media: Ethernet autoselect
        status: no carrier
add net default: gateway 172.16.8.1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
add net fe80::: gateway ::1
add net ff02::: gateway ::1
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat
a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout
Creating and/or trimming log files.
Starting syslogd.
No core dumps found.
Starting rpcbind.
NFS access cache time=60
Setting NIS domain: lmdhw.com.
Startlock order reversal:
ing ypbind.
Sta 1st 0xc5f89990 vm object (standard object) _at_ /usr/src/sys/vm/vm_fault.c:1280
rting amd.
 2nd 0xc0f9b180 page lock (page lock) _at_ /usr/src/sys/vm/vm_fault.c:1299
 3rd 0xc668cbb0 vm object (standard object) _at_ /usr/src/sys/vm/vm_fault.c:1249
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bad0,c08e8615,c08d898b,c0cc700a,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c08d898b,c0cc700a,c553a2b0,c553d648,ebe0bb2c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc700a,c668cbb0,c0cc720f,c553d648,c0cec528,...) at _witness_debugger+0x25
witness_checkorder(c668cbb0,9,c0cec528,4e1,0,...) at witness_checkorder+0x839
_mtx_lock_flags(c668cbb0,0,c0cec528,4e1,5,...) at _mtx_lock_flags+0xc4
vm_fault_copy_entry(c64f34b0,c5586c30,c5faae58,c5faaca8,ebe0bc48,...) at vm_fault_copy_entry+0x228
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 75 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c5faae58,c5faaca8,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 75 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,1,0,517,5,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c5faa1f8,c5f5fc18,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 77 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c5faa1f8,c5f5fc18,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 77 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,391,0,517,3,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c5f5f7e0,c5f50480,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 333 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c5f5f7e0,c5f50480,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 333 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,30,0,517,3,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c15a93a8,c5f50b88,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 429 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c15a93a8,c5f50b88,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 429 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,2,0,517,5,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c5faaa68,c5f50ca8,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 433 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c5faaa68,c5f50ca8,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 433 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,13,0,517,3,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c5f50cf0,c5faa360,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 471 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c5f50cf0,c5faa360,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 471 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,7,0,517,3,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c15a9558,c5faa0d8,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 485 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c15a9558,c5faa0d8,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 485 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,1,0,517,5,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c5fba8b8,c5f50900,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 487 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c5fba8b8,c5f50900,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 487 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,103,0,517,3,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c5fba2d0,c5f5fdc8,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 1005 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c5fba2d0,c5f5fdc8,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 1005 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,6,0,517,5,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c5f50630,c5faa9d8,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 1017 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c5f50630,c5faa9d8,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 1017 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,17,0,517,3,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c5f5f678,c5faae10,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 1063 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c5f5f678,c5faae10,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 1063 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,400,0,517,3,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c5f5fe10,c65b5090,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "MAP ENTRY" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1063 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb14,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb4c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb60,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0cecea9,c0cec528,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b7700,0,2,ebe0bc00,c0b1250a,...) at uma_zalloc_arg+0x34
vm_map_entry_create(c64f34b0,c5586c30,c5f5fe10,c65b5090,ebe0bc48,...) at vm_map_entry_create+0x4d
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x2ba
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "VM OBJECT" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1063 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0baa4,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0badc,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0baf0,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0ced87f,c553a790,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b8a80,0,2,c15b7778,ebe0bc48,...) at uma_zalloc_arg+0x34
vm_object_allocate(0,20,0,517,3,...) at vm_object_allocate+0x2d
vm_fault_copy_entry(c64f34b0,c5586c30,c65b5750,c5f50510,ebe0bc48,...) at vm_fault_copy_entry+0x62
vmspace_fork(c5586c30,ebe0bc48,2,0,0,...) at vmspace_fork+0x57b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x27b
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
lock order reversal:
 1st 0xc0f9a780 page lock (page lock) _at_ /usr/src/sys/vm/vm_fault.c:1299
 2nd 0xc5f8d330 process lock (process lock) _at_ /usr/src/sys/vm/swap_pager.c:208
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb40,c08e8615,c08d898b,c0cc6ff1,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c08d898b,c0cc6ff1,c553a2b0,c5539340,ebe0bb9c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc6ff1,c5f8d330,c0cbf526,c5539340,c0ceb66c,...) at _witness_debugger+0x25
witness_checkorder(c5f8d330,9,c0ceb66c,d0,0,...) at witness_checkorder+0x839
_mtx_lock_flags(c5f8d330,0,c0ceb66c,d0,c5586c30,...) at _mtx_lock_flags+0xc4
swap_reserve_by_uid(945000,0,c5567400,ebe0bc60,c0878318,...) at swap_reserve_by_uid+0x12e
swap_reserve(945000,0,2,0,0,...) at swap_reserve+0x2b
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x298
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
lock order reversal: (sleepable after non-sleepable)
 1st 0xc0f9a780 page lock (page lock) _at_ /usr/src/sys/vm/vm_fault.c:1299
 2nd 0xc0e214f0 proctree (proctree) _at_ /usr/src/sys/kern/kern_fork.c:329
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb7c,c08e8615,c08d898b,c0cc6ff1,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c08d898b,c0cc6ff1,c553a2b0,c5539068,ebe0bbd8,...) at kdb_backtrace+0x29
_witness_debugger(c0cc6ff1,c0e214f0,c0cbfd3f,c5539068,c0cbbe2a,...) at _witness_debugger+0x25
witness_checkorder(c0e214f0,1,c0cbbe2a,149,0,...) at witness_checkorder+0x839
_sx_slock(c0e214f0,0,c0cbbe2a,149,0,...) at _sx_slock+0x85
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x320
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "4096" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1127 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb30,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb68,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb7c,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0c5929c,c0cbbe2a,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b0700,0,102,2,37d,...) at uma_zalloc_arg+0x34
malloc(abc,c0db7000,102,1869f,37d,...) at malloc+0xe4
sigacts_alloc(c5f8d880,0,c0cbbe2a,1de,0,...) at sigacts_alloc+0x23
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x858
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
uma_zalloc_arg: zone "256" with the following non-sleepable locks held:
exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1127 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb00,c08e8615,c0cec528,513,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c0cec528,513,ffffffff,c0f61214,ebe0bb38,...) at kdb_backtrace+0x29
_witness_debugger(c0cc659d,ebe0bb4c,4,1,0,...) at _witness_debugger+0x25
witness_warn(5,0,c0cec231,c0c56d98,109,...) at witness_warn+0x1fd
uma_zalloc_arg(c15b1700,0,102,2,37d,...) at uma_zalloc_arg+0x34
malloc(b8,c0db2b84,102,c08923ea,37d,...) at malloc+0xe4
fdinit(c65aa200,ebe0bc00,c08a2e55,c6675aa8,c0cc0bde,...) at fdinit+0x28
fdcopy(c65aa200,0,c0cbbe2a,1de,0,...) at fdcopy+0x21
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x893
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
lock order reversal: (sleepable after non-sleepable)
 1st 0xc0f9a780 page lock (page lock) _at_ /usr/src/sys/vm/vm_fault.c:1299
 2nd 0xc65aa22c filedesc structure (filedesc structure) _at_ /usr/src/sys/kern/kern_descrip.c:1580
KDB: stack backtrace:
db_trace_self_wrapper(c0cc4017,ebe0bb30,c08e8615,c08d898b,c0cc6ff1,...) at db_trace_self_wrapper+0x26
kdb_backtrace(c08d898b,c0cc6ff1,c553a2b0,c553d308,ebe0bb8c,...) at kdb_backtrace+0x29
_witness_debugger(c0cc6ff1,c65aa22c,c0cbb557,c553d308,c0cbb30f,...) at _witness_debugger+0x25
witness_checkorder(c65aa22c,9,c0cbb30f,62c,0,...) at witness_checkorder+0x839
_sx_xlock(c65aa22c,0,c0cbb30f,62c,37d,...) at _sx_xlock+0x85
fdinit(c65aa200,ebe0bc00,c08a2e55,c6675aa8,c0cc0bde,...) at fdinit+0x6c
fdcopy(c65aa200,0,c0cbbe2a,1de,0,...) at fdcopy+0x21
fork1(c5f92000,14,0,ebe0bc78,c5f92000,...) at fork1+0x893
fork(c5f92000,ebe0bcf8,c0cfcac4,c0cc7889,c5f8d2a8,...) at fork+0x29
syscall(ebe0bd38) at syscall+0x220
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (2, FreeBSD ELF32, fork), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
System call fork returning with the following locks held:
exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1127 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
panic: witness_warn
cpuid = 1
KDB: enter: panic
[ thread pid 892 tid 100089 ]
Stopped at      kdb_enter+0x3a: movl    $0,kdb_why
db> bt
Tracing pid 892 tid 100089 td 0xc5f92000
kdb_enter(c0cc0b06,c0cc0b06,c0c68e9c,ebe0bc54,1,...) at kdb_enter+0x3a
panic(c0c68e9c,c0c434fd,0,0,0,...) at panic+0x136
witness_warn(2,0,c0cfcac4,c0cbbefd,c5f8d2a8,...) at witness_warn+0x1e9
syscall(ebe0bd38) at syscall+0x2d8
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (893), eip = 0x2811d37b, esp = 0xbfbfe9cc, ebp = 0xbfbfe9d8 ---
db> show locks
exclusive sleep mutex page lock (page lock) r = 2047 (0xc0f9a780) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1127 (0xc0f9ae00) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
exclusive sleep mutex page lock (page lock) r = 1569 (0xc0f9b180) locked _at_ /usr/src/sys/vm/vm_fault.c:1299
db> 


Please let me know if additional information might be useful.

I should be able to try various experiments & patches.

As I type, I'm still building the kernel for CURRENT as of r207433 on
my laptop; I should be able to report whether or not it exhibits similar
behavior.

Peace,
david
-- 
David H. Wolfskill				david_at_catwhisker.org
Depriving a girl or boy of an opportunity for education is evil.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.

Received on Fri Apr 30 2010 - 14:20:26 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:03 UTC