I've been working on a patch to fix the vnode locking bugs in NFS server code in -current and stumbled across a problem that appears to be unrelated to NFS. As a torture test, I've NFS mounted /tmp/obj on /usr/obj and run "make buildworld". The first time I tried this, I didn't have any problems, so I tried again with -j10, which wedged all the nfsd processes. After updating to the latest source, I ran "make buildworld" again and got another hang, even without the -j option. The nfsd processes all hang while attempting to lock a vnode, but the reason I don't think this is related to NFS is that the vnode lock is held by bufdaemon, which is sleeping on runningbufreq. The vnode in question this time around is 0xc9ba27fc, which corresponds to /tmp/obj/usr/src/gnu/usr.bin/cc/cc1/cc1. The stack for the bufdaemon thread holding the lock is: Proc 0xc6153000 [SLPQ wdrain c05ce978][SLP] bufdaemon mi_switch(c60d8260,44,c050c1cf,ca,1) at mi_switch+0x210 msleep(c05ce978,c05ce980,44,c0511486,0) at msleep+0x432 bwrite(d28ce2e0,0,c0511368,697,137e400) at bwrite+0x442 vfs_bio_awrite(d28ce2e0,0,c0511368,87b,0) at vfs_bio_awrite+0x221 flushbufqueues(0,0,c0511368,10e,64) at flushbufqueues+0x17d buf_daemon(0,e0a92d48,c0509769,310,0) at buf_daemon+0xdc fork_exit(c0363240,0,e0a92d48) at fork_exit+0xc0 fork_trampoline() at fork_trampoline+0x1a What is puzzling is why this process is sleeping here. It appears that maybe a wakeup didn't happen. This machine has 1 GB of RAM, so I don't think memory pressure should be a cause. Here's the source at bwrite+0x442 (kgdb) list *bwrite+0x442 0xc0360f82 is in bwrite (/usr/src/sys/kern/vfs_bio.c:362). 357 waitrunningbufspace(void) 358 { 359 mtx_lock(&rbreqlock); 360 while (runningbufspace > hirunningspace) { 361 ++runningbufreq; 362 msleep(&runningbufreq, &rbreqlock, PVM, "wdrain", 0); 363 } 364 mtx_unlock(&rbreqlock); 365 } 366 which is part of the waitrunningbufspace() inline routine. Bufdaemon shouldn't be sleeping here because (kgdb) print runningbufspace $1 = 548864 (kgdb) print hirunningspace $2 = 1048576 (kgdb) print runningbufreq $3 = 3 Once all the nfsd processes block, I get an NFS server not responding message and the buildworld freezes, but the system is otherwise responsive. Here is all the stuff I thought to print from DDB: db> ps pid proc addr uid ppid pgrp flag stat wmesg wchan cmd 4160 c6a5db40 e6e3d000 0 4159 559 0004002 norm[SLPQ nfsrcvlk c6640e04][SLP] ld 4159 c66a73c0 e6d8a000 0 4158 559 0004002 norm[SLPQ wait c66a73c0][SLP] cc 4158 c76ad000 e6e7d000 0 4141 559 0004002 norm[SLPQ wait c76ad000][SLP] sh 4141 c76adb40 e6e87000 0 3254 559 0004002 norm[SLPQ wait c76adb40][SLP] make 3254 c66a71e0 e6d89000 0 3253 559 0004002 norm[SLPQ wait c66a71e0][SLP] sh 3253 c76abb40 e6e7b000 0 1779 559 0004002 norm[SLPQ wait c76abb40][SLP] make 1779 c76ab3c0 e6e6f000 0 1778 559 0004002 norm[SLPQ wait c76ab3c0][SLP] sh 1778 c686dd20 e6e2b000 0 1732 559 0004002 norm[SLPQ wait c686dd20][SLP] make 1732 c76ad960 e6e86000 0 1731 559 0004002 norm[SLPQ wait c76ad960][SLP] sh 1731 c76abd20 e6e7c000 0 1730 559 0004002 norm[SLPQ wait c76abd20][SLP] make 1730 c6661d20 e6d5e000 0 98448 559 0004002 norm[SLPQ wait c6661d20][SLP] sh 898 c76ab780 e6e75000 0 0 0 0000204 norm[SLPQ nfsrcvlk c6640e04][SLP] nfsiod 10 897 c6a5c3c0 e6d82000 0 0 0 0000204 norm[SLPQ sbwait c6689b64][SLP] nfsiod 9 98448 c76ad780 e6e85000 0 98447 559 0004002 norm[SLPQ wait c76ad780][SLP] make 98447 c6a5d5a0 e6e3a000 0 604 559 0004002 norm[SLPQ wait c6a5d5a0][SLP] sh 97622 c66a8960 e6dbc000 0 0 0 0000204 norm[SLPQ nfsrcvlk c6640e04][SLP] nfsiod 6 97621 c6a5cd20 e6d87000 0 0 0 0000204 norm[SLPQ nfsrcvlk c6640e04][SLP] nfsiod 5 93797 c6a5d000 e6e37000 0 0 0 0000204 norm[SLPQ nfsrcvlk c6640e04][SLP] nfsiod 4 604 c686c5a0 e6dd1000 0 602 559 0004002 norm[SLPQ wait c686c5a0][SLP] make 602 c686d000 e6dd6000 0 559 559 0004002 norm[SLPQ wait c686d000][SLP] sh 559 c66a8d20 e6dbe000 0 554 559 0004002 norm[SLPQ wait c66a8d20][SLP] make 554 c686c780 e6dd2000 0 553 554 0004002 norm[SLPQ pause e6dd2000][SLP] csh 553 c686c960 e6dd3000 1001 547 553 0004102 norm[SLPQ wait c686c960][SLP] su 547 c686d5a0 e6dd9000 1001 546 547 0004002 norm[SLPQ pause e6dd9000][SLP] csh 546 c66a83c0 e6db9000 1001 544 544 0000100 norm[CVQ select c05ce574][SLP] sshd 544 c66a81e0 e6db8000 0 416 544 0000100 norm[SLPQ sbwait c6862264][SLP] sshd 543 c66a7960 e6d8d000 0 1 543 0004002 norm[SLPQ ttyin c639de10][SLP] getty 542 c665d5a0 e6d03000 0 1 542 0004002 norm[SLPQ ttyin c66df210][SLP] getty 541 c66a85a0 e6dba000 0 1 541 0004002 norm[SLPQ ttyin c66df810][SLP] getty 540 c665d000 e6d00000 0 1 540 0004002 norm[SLPQ ttyin c66dfe10][SLP] getty 539 c665d1e0 e6d01000 0 1 539 0004002 norm[SLPQ ttyin c66e0410][SLP] getty 538 c66a7d20 e6d8f000 0 1 538 0004002 norm[SLPQ ttyin c6147410][SLP] getty 537 c6661b40 e6d5d000 0 1 537 0004002 norm[SLPQ ttyin c60bfa10][SLP] getty 536 c63a33c0 e4b00000 0 1 536 0004002 norm[SLPQ ttyin c60c0e10][SLP] getty 524 c6155b40 e4afb000 0 1 524 0000000 norm[CVQ select c05ce574][SLP] inetd 503 c66615a0 e6d5a000 0 1 503 0000000 norm[CVQ select c05ce574][SLP] moused 486 c66a7000 e6d88000 65534 1 486 0004002 norm[RUNQ] setiathome 478 c66a75a0 e6d8b000 0 1 478 0000000 norm[SLPQ nanslp c05a976c][SLP] upslog 477 c66a7780 e6d8c000 66 475 475 0000100 norm[SLPQ nanslp c05a976c][SLP] upsmon 475 c66a7b40 e6d8e000 0 1 475 0000000 norm[SLPQ piperd c63c0370][SLP] upsmon 473 c66a8000 e6d90000 66 1 473 0000101 norm[CVQ select c05ce574][SLP] upsd 468 c6661000 e6d57000 66 1 468 0000101 norm[SLPQ ttyin c60bf010][SLP] newapc 463 c665db40 e6d06000 0 1 463 0000000 norm[CVQ select c05ce574][SLP] healthd 440 c66611e0 e6d58000 0 1 440 0000000 norm[SLPQ nanslp c05a976c][SLP] cron 425 c665dd20 e6d07000 25 1 425 0000100 norm[SLPQ pause e6d07000][SLP] sendmail 422 c665d960 e6d05000 0 1 422 0000100 norm[CVQ select c05ce574][SLP] sendmail 416 c665d780 e6d04000 0 1 416 0000100 norm[CVQ select c05ce574][SLP] sshd 400 c63a3d20 e4b05000 0 1 400 0000000 norm[CVQ select c05ce574][SLP] usbd 383 c665d3c0 e6d02000 0 1 383 0000000 norm[CVQ select c05ce574][SLP] ntpd 368 c66613c0 e6d59000 0 1 368 0000000 norm[CVQ select c05ce574][SLP] lpd 339 c6661780 e6d5b000 0 334 334 0000000 norm[SLPQ ufs c9ba28b8][SLP] nfsd 338 c63a31e0 e4aff000 0 334 334 0000000 norm[SLPQ ufs c9ba28b8][SLP] nfsd 336 c61553c0 e4ad0000 0 334 334 0000000 norm[SLPQ ufs c9ba28b8][SLP] nfsd 335 c6155780 e4ad2000 0 334 334 0000000 norm[SLPQ ufs c9ba28b8][SLP] nfsd 334 c6155d20 e4afc000 0 1 334 0000000 norm[CVQ select c05ce574][SLP] nfsd 328 c61555a0 e4ad1000 0 1 328 0000000 norm[CVQ select c05ce574][SLP] mountd 266 c6155960 e4ad3000 0 1 266 0000000 norm[CVQ select c05ce574][SLP] rpcbind 250 c63a3000 e4afe000 0 1 250 0000000 norm[CVQ select c05ce574][SLP] syslogd 46 c63a35a0 e4b01000 0 0 0 0000204 norm[SLPQ nfsrcvlk c6640e04][SLP] nfsiod 3 45 c63a3780 e4b02000 0 0 0 0000204 norm[SLPQ nfsrcvlk c6640e04][SLP] nfsiod 2 44 c63a3960 e4b03000 0 0 0 0000204 norm[SLPQ nfsrcvlk c6640e04][SLP] nfsiod 1 43 c63a3b40 e4b04000 0 0 0 0000204 norm[SLPQ nfsrcvlk c6640e04][SLP] nfsiod 0 42 c60d7b40 e0ab1000 0 0 0 0000204 norm[SLPQ vlruwt c60d7b40][SLP] vnlru 41 c60d7d20 e0ab2000 0 0 0 0000204 norm[SLPQ wdrain c05ce978][SLP] syncer 40 c6153000 e4ac6000 0 0 0 0000204 norm[SLPQ wdrain c05ce978][SLP] bufdaemon 39 c61531e0 e4ac7000 0 0 0 000020c norm[RUNQ] pagezero 9 c61533c0 e4ac8000 0 0 0 0000204 norm[SLPQ psleep c05d67f4][SLP] vmdaemon 8 c61535a0 e4ac9000 0 0 0 0000204 norm[SLPQ psleep c05d67e0][SLP] pagedaemon 38 c6153780 e4aca000 0 0 0 0000204 new [IWAIT] irq8: rtc 37 c6153960 e4acb000 0 0 0 0000204 new [IWAIT] irq0: clk 36 c6153b40 e4acc000 0 0 0 0000204 new [IWAIT] irq12: psm0 35 c6153d20 e4acd000 0 0 0 0000204 norm[CPU 0] irq1: atkbd0 34 c6155000 e4ace000 0 0 0 0000204 norm[IWAIT] irq7: ppc0 33 c61551e0 e4acf000 0 0 0 0000204 new [IWAIT] irq3: sio1 32 c60015a0 e0a7a000 0 0 0 0000204 new [IWAIT] irq4: sio0 31 c6001780 e0a7b000 0 0 0 0000204 norm[IWAIT] swi0: tty:sio 30 c6001960 e0a7c000 0 0 0 0000204 norm[IWAIT] irq6: fdc0 29 c6001b40 e0a7d000 0 0 0 0000204 norm[IWAIT] irq11: fxp0 28 c6001d20 e0a7e000 0 0 0 0000204 norm[SLPQ usbevt c612f210][SLP] usb1 27 c60d7000 e0a84000 0 0 0 0000204 norm[SLPQ usbtsk c05a0058][SLP] usbtask 26 c60d71e0 e0a85000 0 0 0 0000204 norm[SLPQ usbevt c613c210][SLP] usb0 25 c60d73c0 e0a86000 0 0 0 0000204 new [IWAIT] irq10: uhci0 uhci1 24 c60d75a0 e0a87000 0 0 0 0000204 norm[IWAIT] irq15: ata1 ahc0 --More-- 23 c60d7780 e0a88000 0 0 0 0000204 new [IWAIT] irq14: ata0 7 c60d7960 e0a89000 0 0 0 0000204 norm[SLPQ actask c06fb0cc][SLP] acpi_task2 6 c20e71e0 df048000 0 0 0 0000204 norm[SLPQ actask c06fb0cc][SLP] acpi_task1 5 c20e73c0 df049000 0 0 0 0000204 norm[SLPQ actask c06fb0cc][SLP] acpi_task0 22 c20e75a0 df04a000 0 0 0 0000204 new [IWAIT] irq9: acpi0 21 c20e7780 df04b000 0 0 0 0000204 new [IWAIT] irq13: 20 c20e7960 df04c000 0 0 0 0000204 new [IWAIT] swi6: task queue 19 c20e7b40 df04d000 0 0 0 0000204 norm[IWAIT] swi3: cambio 18 c20e7d20 df075000 0 0 0 0000204 new [IWAIT] swi2: camnet 17 c6001000 e0a77000 0 0 0 0000204 new [IWAIT] swi5:+ 16 c60011e0 e0a78000 0 0 0 0000204 norm[IWAIT] swi6: acpitaskq 15 c60013c0 e0a79000 0 0 0 0000204 norm[SLPQ sleep c0593ea0][SLP] random 4 c20e0000 deff1000 0 0 0 0000204 norm[SLPQ g_down c05a25b8][SLP] g_down 3 c20e01e0 df040000 0 0 0 0000204 norm[SLPQ g_up c05a25b4][SLP] g_up 2 c20e03c0 df041000 0 0 0 0000204 norm[SLPQ g_events c05a25ac][SLP] g_event 14 c20e05a0 df042000 0 0 0 0000204 new [IWAIT] swi4: vm 13 c20e0780 df043000 0 0 0 000020c norm[RUNQ] swi7: tty:sio clock 12 c20e0960 df044000 0 0 0 0000204 norm[IWAIT] swi1: net 11 c20e0b40 df045000 0 0 0 000020c norm[Can run] idle 1 c20e0d20 df046000 0 0 1 0004200 norm[SLPQ wait c20e0d20][SLP] init 10 c20e7000 df047000 0 0 0 0000204 norm[CVQ ktrace c05a5bf4][SLP] ktrace --More-- 0 c05a2720 c072a000 0 0 0 0000200 norm[SLPQ sched c05a2720][SLP] swapper db> tr 335 mi_switch(c20e8390,50,c050c1cf,ca,c20e8390) at mi_switch+0x210 msleep(c9ba28b8,c05a7d08,50,c0515f0d,0) at msleep+0x432 acquire(df059930,1000040,600,ef,c20e8390) at acquire+0x9e lockmgr(c9ba28b8,1010002,c9ba27fc,c20e8390,df05994c) at lockmgr+0x387 vop_stdlock(df059974,df059958,c0455358,df059974,df059994) at vop_stdlock+0x2c vop_defaultop(df059974,df059994,c037feb9,df059974,246) at vop_defaultop+0x18 ufs_vnoperate(df059974,246,c051fb05,0,df059994) at ufs_vnoperate+0x18 vn_lock(c9ba27fc,10002,c20e8390,163,c05b2320) at vn_lock+0xe9 vget(c9ba27fc,10002,c20e8390,7f,c20e8390) at vget+0xd6 ufs_ihashget(c63a6b00,137b94,2,df059a90,c05ce4dc) at ufs_ihashget+0xdc ffs_vget(c6393000,137b94,2,df059a90,0) at ffs_vget+0x3e ufs_fhtovp(c6393000,df059b38,df059b1c,c6394000,df059c5c) at ufs_fhtovp+0x37 ffs_fhtovp(c6393000,df059b38,df059b1c,df059b2c,c05b1ea0) at ffs_fhtovp+0x47 nfsrv_commit(c6b2bc00,c6630c00,c20e8390,df059c8c,0) at nfsrv_commit+0x150 nfssvc_nfsd(c20e8390,0,c05199d2,96,df059ce0) at nfssvc_nfsd+0x3aa nfssvc(c20e8390,df059d10,c0526328,3fb,2) at nfssvc+0x17d syscall(2f,2f,2f,0,4) at syscall+0x26e Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x480b7bb3, esp = 0xbfbffa1c, ebp = 0xbfbffa38 --- db> tr 336 mi_switch(c20e85f0,50,c050c1cf,ca,c20e85f0) at mi_switch+0x210 msleep(c9ba28b8,c05a7d08,50,c0515f0d,0) at msleep+0x432 acquire(df05f930,1000040,600,ef,c20e85f0) at acquire+0x9e lockmgr(c9ba28b8,1010002,c9ba27fc,c20e85f0,df05f94c) at lockmgr+0x387 vop_stdlock(df05f974,df05f958,c0455358,df05f974,df05f994) at vop_stdlock+0x2c vop_defaultop(df05f974,df05f994,c037feb9,df05f974,246) at vop_defaultop+0x18 ufs_vnoperate(df05f974,246,c051fb05,0,df05f994) at ufs_vnoperate+0x18 vn_lock(c9ba27fc,10002,c20e85f0,163,c05b2320) at vn_lock+0xe9 vget(c9ba27fc,10002,c20e85f0,7f,c20e85f0) at vget+0xd6 ufs_ihashget(c63a6b00,137b94,2,df05fa90,c05ce4dc) at ufs_ihashget+0xdc ffs_vget(c6393000,137b94,2,df05fa90,0) at ffs_vget+0x3e ufs_fhtovp(c6393000,df05fb38,df05fb1c,c6394c00,df05fc5c) at ufs_fhtovp+0x37 ffs_fhtovp(c6393000,df05fb38,df05fb1c,df05fb2c,c05b1ea0) at ffs_fhtovp+0x47 nfsrv_commit(c6b5c500,c6630c00,c20e85f0,df05fc8c,0) at nfsrv_commit+0x150 nfssvc_nfsd(c20e85f0,0,c05199d2,96,df05fce0) at nfssvc_nfsd+0x3aa nfssvc(c20e85f0,df05fd10,c0526328,3fb,2) at nfssvc+0x17d syscall(2f,2f,2f,1,4) at syscall+0x26e Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x480b7bb3, esp = 0xbfbffa1c, ebp = 0xbfbffa38 --- db> tr 338 mi_switch(c61565f0,50,c050c1cf,ca,c61565f0) at mi_switch+0x210 msleep(c9ba28b8,c05a7d08,50,c0515f0d,0) at msleep+0x432 acquire(e4ae5930,1000040,600,ef,c61565f0) at acquire+0x9e lockmgr(c9ba28b8,1010002,c9ba27fc,c61565f0,e4ae594c) at lockmgr+0x387 vop_stdlock(e4ae5974,e4ae5958,c0455358,e4ae5974,e4ae5994) at vop_stdlock+0x2c vop_defaultop(e4ae5974,e4ae5994,c037feb9,e4ae5974,246) at vop_defaultop+0x18 ufs_vnoperate(e4ae5974,246,c051fb05,0,e4ae5994) at ufs_vnoperate+0x18 vn_lock(c9ba27fc,10002,c61565f0,163,c05b2320) at vn_lock+0xe9 vget(c9ba27fc,10002,c61565f0,7f,c61565f0) at vget+0xd6 ufs_ihashget(c63a6b00,137b94,2,e4ae5a90,c05ce4dc) at ufs_ihashget+0xdc ffs_vget(c6393000,137b94,2,e4ae5a90,0) at ffs_vget+0x3e ufs_fhtovp(c6393000,e4ae5b38,e4ae5b1c,c6394800,e4ae5c5c) at ufs_fhtovp+0x37 ffs_fhtovp(c6393000,e4ae5b38,e4ae5b1c,e4ae5b2c,c05b1ea0) at ffs_fhtovp+0x47 nfsrv_commit(c6b23900,c6630c00,c61565f0,e4ae5c8c,0) at nfsrv_commit+0x150 nfssvc_nfsd(c61565f0,0,c05199d2,96,e4ae5ce0) at nfssvc_nfsd+0x3aa nfssvc(c61565f0,e4ae5d10,c0526328,3fb,2) at nfssvc+0x17d syscall(2f,2f,2f,2,4) at syscall+0x26e Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x480b7bb3, esp = 0xbfbffa1c, ebp = 0xbfbffa38 --- db> tr 339 mi_switch(c665e980,50,c050c1cf,ca,c665e980) at mi_switch+0x210 msleep(c9ba28b8,c05a7d08,50,c0515f0d,0) at msleep+0x432 acquire(e6d23930,1000040,600,ef,c665e980) at acquire+0x9e lockmgr(c9ba28b8,1010002,c9ba27fc,c665e980,e6d2394c) at lockmgr+0x387 vop_stdlock(e6d23974,e6d23958,c0455358,e6d23974,e6d23994) at vop_stdlock+0x2c vop_defaultop(e6d23974,e6d23994,c037feb9,e6d23974,246) at vop_defaultop+0x18 ufs_vnoperate(e6d23974,246,c051fb05,0,e6d23994) at ufs_vnoperate+0x18 vn_lock(c9ba27fc,10002,c665e980,163,c05b2320) at vn_lock+0xe9 vget(c9ba27fc,10002,c665e980,7f,c665e980) at vget+0xd6 ufs_ihashget(c63a6b00,137b94,2,e6d23a90,c05ce4dc) at ufs_ihashget+0xdc ffs_vget(c6393000,137b94,2,e6d23a90,0) at ffs_vget+0x3e ufs_fhtovp(c6393000,e6d23b38,e6d23b1c,c6394600,e6d23c5c) at ufs_fhtovp+0x37 ffs_fhtovp(c6393000,e6d23b38,e6d23b1c,e6d23b2c,c05b1ea0) at ffs_fhtovp+0x47 nfsrv_commit(c6b3ba00,c6630c00,c665e980,e6d23c8c,0) at nfsrv_commit+0x150 nfssvc_nfsd(c665e980,0,c05199d2,96,e6d23ce0) at nfssvc_nfsd+0x3aa nfssvc(c665e980,e6d23d10,c0526328,3fb,2) at nfssvc+0x17d syscall(2f,2f,2f,3,4) at syscall+0x26e Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x480b7bb3, esp = 0xbfbffa1c, ebp = 0xbfbffa38 --- db> tr Debugger(c0523a5c,4,1,0,1) at Debugger+0x54 scgetc(c20c1800,2,1d7,c05ac028,c615e440) at scgetc+0x445 sckbdevent(c05d6ea0,0,c20c1800,c05a70e0,8) at sckbdevent+0x1e0 atkbd_intr(c05d6ea0,0,df068d0c,c03093b2,c05d6ea0) at atkbd_intr+0x2c atkbd_isa_intr(c05d6ea0,0,c05098be,218,c6153d20) at atkbd_isa_intr+0x21 ithread_loop(c615d080,df068d48,c0509769,310,0) at ithread_loop+0x182 fork_exit(c0309230,c615d080,df068d48) at fork_exit+0xc0 fork_trampoline() at fork_trampoline+0x1a --- trap 0x1, eip = 0, esp = 0xdf068d7c, ebp = 0 --- db> sho lockedvnod Locked vnodes 0xc6625b68: tag syncer, type VNON, usecount 1, writecount 0, refcount 0, lock type syncer: EXCL (count 1) by thread 0xc60d8390 0xc9ba27fc: tag ufs, type VREG, usecount 4, writecount 0, refcount 194, flags (VV_OBJBUF), lock type ufs: EXCL (count 1) by thread 0xc60d8260 with 4 pending ino 1276820, on dev da0s1a (4, 15) 0xc9ba1db0: tag ufs, type VREG, usecount 1, writecount 0, refcount 185, flags (VV_OBJBUF), lock type ufs: EXCL (count 1) by thread 0xc60d8390 ino 1276823, on dev da0s1a (4, 15) 0xc9ba1c8c: tag nfs, type VREG, usecount 1, writecount 1, refcount 368, flags (VV_OBJBUF), lock type nfs: EXCL (count 1) by thread 0xc66a9980 fileid 1276823 fsid 0x500ff04 db> sho thread 0xc60d8390 Proc 0xc60d7d20 [SLPQ wdrain c05ce978][SLP] syncer mi_switch(c60d8390,44,c050c1cf,ca,e0a95aa8) at mi_switch+0x210 msleep(c05ce978,c05ce980,44,c0511486,0) at msleep+0x432 bwrite(d2804c38,e0a95b4c,c0369751,d2804c38,d2804d68) at bwrite+0x442 bawrite(d2804c38,d2804d68,8,3c4,20020080) at bawrite+0x1c cluster_wbuild(c9ba1db0,4000,1d,0,2) at cluster_wbuild+0x6d1 vfs_bio_awrite(d2a81e20,0,c051f4ff,d7,c0370307) at vfs_bio_awrite+0x1f9 ffs_fsync(e0a95be4,c05051b3,c057e520,c9ba1db0,c20c8e80) at ffs_fsync+0x2b2 VOP_FSYNC(c9ba1db0,c20c8e80,3,c60d8390,0) at VOP_FSYNC+0x5b ffs_sync(c6393000,3,c20c8e80,c60d8390,c6393000) at ffs_sync+0x184 sync_fsync(e0a95c94,c05051b3,c057e520,c6625b68,c20c8e80) at sync_fsync+0x16a VOP_FSYNC(c6625b68,c20c8e80,3,c60d8390,0) at VOP_FSYNC+0x5b sched_sync(0,e0a95d48,c0509769,310,0) at sched_sync+0x16b fork_exit(c0373080,0,e0a95d48) at fork_exit+0xc0 fork_trampoline() at fork_trampoline+0x1a --- trap 0x1, eip = 0, esp = 0xe0a95d7c, ebp = 0 --- db> sho thread 0xc60d8260 Proc 0xc6153000 [SLPQ wdrain c05ce978][SLP] bufdaemon mi_switch(c60d8260,44,c050c1cf,ca,1) at mi_switch+0x210 msleep(c05ce978,c05ce980,44,c0511486,0) at msleep+0x432 bwrite(d28ce2e0,0,c0511368,697,137e400) at bwrite+0x442 vfs_bio_awrite(d28ce2e0,0,c0511368,87b,0) at vfs_bio_awrite+0x221 flushbufqueues(0,0,c0511368,10e,64) at flushbufqueues+0x17d buf_daemon(0,e0a92d48,c0509769,310,0) at buf_daemon+0xdc fork_exit(c0363240,0,e0a92d48) at fork_exit+0xc0 fork_trampoline() at fork_trampoline+0x1a --- trap 0x1, eip = 0, esp = 0xe0a92d7c, ebp = 0 --- db> sho thread 0xc60d8390 Proc 0xc60d7d20 [SLPQ wdrain c05ce978][SLP] syncer mi_switch(c60d8390,44,c050c1cf,ca,e0a95aa8) at mi_switch+0x210 msleep(c05ce978,c05ce980,44,c0511486,0) at msleep+0x432 bwrite(d2804c38,e0a95b4c,c0369751,d2804c38,d2804d68) at bwrite+0x442 bawrite(d2804c38,d2804d68,8,3c4,20020080) at bawrite+0x1c cluster_wbuild(c9ba1db0,4000,1d,0,2) at cluster_wbuild+0x6d1 vfs_bio_awrite(d2a81e20,0,c051f4ff,d7,c0370307) at vfs_bio_awrite+0x1f9 ffs_fsync(e0a95be4,c05051b3,c057e520,c9ba1db0,c20c8e80) at ffs_fsync+0x2b2 VOP_FSYNC(c9ba1db0,c20c8e80,3,c60d8390,0) at VOP_FSYNC+0x5b ffs_sync(c6393000,3,c20c8e80,c60d8390,c6393000) at ffs_sync+0x184 sync_fsync(e0a95c94,c05051b3,c057e520,c6625b68,c20c8e80) at sync_fsync+0x16a VOP_FSYNC(c6625b68,c20c8e80,3,c60d8390,0) at VOP_FSYNC+0x5b sched_sync(0,e0a95d48,c0509769,310,0) at sched_sync+0x16b fork_exit(c0373080,0,e0a95d48) at fork_exit+0xc0 fork_trampoline() at fork_trampoline+0x1a --- trap 0x1, eip = 0, esp = 0xe0a95d7c, ebp = 0 --- db> sho thread 0xc66a9980 Proc 0xc6a5db40 [SLPQ nfsrcvlk c6640e04][SLP] ld mi_switch(c66a9980,53,c050c1cf,ed,c66a9980) at mi_switch+0x210 msleep(c6640e04,0,153,c0519238,0) at msleep+0x432 nfs_rcvlock(c9ba8800,0,0,c6640e00,0) at nfs_rcvlock+0x7d nfs_reply(c9ba8800,c615bc40,c20fe100,c9ba8800,1c) at nfs_reply+0x2b nfs_request(c9ba1c8c,c20fc000,7,c66a9980,c667a380) at nfs_request+0x2dc nfs_writerpc(c9ba1c8c,e6daba2c,c667a380,e6daba1c,e6daba18) at nfs_writerpc+0x259 nfs_doio(d286c360,c667a380,c66a9980,0,c9ba1c8c) at nfs_doio+0x58e nfs_strategy(e6dabaa0,1,c0511368,35e,16c) at nfs_strategy+0xd7 bwrite(d286c360,16c,0,2000,c66a9980) at bwrite+0x3af nfs_write(e6dabbc8,c04f43a8,c057e3a0,c9ba1c8c,e6dabc7c) at nfs_write+0x86b VOP_WRITE(c9ba1c8c,e6dabc7c,1,c667a380,0) at VOP_WRITE+0x5b vn_write(c668bb04,e6dabc7c,c667a380,0,c66a9980) at vn_write+0x178 dofilewrite(c66a9980,c668bb04,7,80e1a00,14) at dofilewrite+0xe8 write(c66a9980,e6dabd10,c0526328,3fb,3) at write+0x69 syscall(2f,2f,2f,80d9b00,80d9b00) at syscall+0x26e Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (4, FreeBSD ELF32, write), eip = 0x80a80df, esp = 0xbfbff12c, ebp = 0xbfbff148 --- db> panic panic: from debugger syncing disks, buffers remaining... VOP_LOCK: 0xc66846d8 is not locked but should be VOP_GETVOBJECT: 0xc66846d8 is not locked but should be VOP_GETVOBJECT: 0xc66846d8 is not locked but should be VOP_UNLOCK: 0xc66846d8 is not locked but should be VOP_LOCK: 0xc662536c is not locked but should be VOP_FSYNC: 0xc662536c is not locked but should be VOP_FSYNC: 0xc662536c is not locked but should be VOP_UNLOCK: 0xc662536c is not locked but should be VOP_LOCK: 0xc662536c is not locked but should be VOP_INACTIVE: 0xc662536c is not locked but should be VOP_UNLOCK: 0xc662536c is not locked but should be VOP_LOCK: 0xc6632124 is not locked but should be VOP_FSYNC: 0xc6632124 is not locked but should be VOP_FSYNC: 0xc6632124 is not locked but should be VOP_UNLOCK: 0xc6632124 is not locked but should be VOP_LOCK: 0xc6645000 is not locked but should be VOP_FSYNC: 0xc6645000 is not locked but should be VOP_FSYNC: 0xc6645000 is not locked but should be VOP_UNLOCK: 0xc6645000 is not locked but should be VOP_LOCK: 0xc66af248 is not locked but should be VOP_FSYNC: 0xc66af248 is not locked but should be VOP_FSYNC: 0xc66af248 is not locked but should be VOP_UNLOCK: 0xc66af248 is not locked but should be VOP_LOCK: 0xc66bdc8c is not locked but should be VOP_FSYNC: 0xc66bdc8c is not locked but should be VOP_FSYNC: 0xc66bdc8c is not locked but should be VOP_UNLOCK: 0xc66bdc8c is not locked but should be VOP_LOCK: 0xc66bdc8c is not locked but should be VOP_INACTIVE: 0xc66bdc8c is not locked but should be VOP_UNLOCK: 0xc66bdc8c is not locked but should be VOP_LOCK: 0xc681f248 is not locked but should be VOP_FSYNC: 0xc681f248 is not locked but should be VOP_FSYNC: 0xc681f248 is not locked but should be VOP_UNLOCK: 0xc681f248 is not locked but should be VOP_LOCK: 0xc681f248 is not locked but should be VOP_INACTIVE: 0xc681f248 is not locked but should be VOP_UNLOCK: 0xc681f248 is not locked but should be panic: KSE not on run queue Uptime: 1h23m55s Dumping 1023 MB 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 256 272 288 304 320 336 352 368 384 400 416 432 448 464 480 496 512 528 544 560 576 592 608 624 640 656 672 688 704 720 736 752 768 784 800 816 832 848 864 880 896 912 928 944 960 976 992 1008 Dump complete Terminate ACPI Automatic reboot in 15 seconds - press a key on the console to abort Rebooting...Received on Tue May 06 2003 - 19:58:44 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:37:06 UTC