PANIC: vrele: negative ref cnt

Yesterday I got the above panic and I think I have a guess about what prompts 
the bug.  First:

---
panic: vrele: negative ref cnt
at line 2015 in file ../../../kern/vfs_subr.c
Debugger("panic")
Uptime: 16h36m17s
Dumping 1023 MB
 16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 256 272 288 304 320 336 
352 368 384 400 416 432 448 464 480 496 512 528 544 560 576 592 608 624 640 
656 672 688 704 720 736 752 768 784 800 816 832 848 864 880 896 912 928 944 
960 976 992 1008
---
(kgdb) where
#0  doadump () at ../../../kern/kern_shutdown.c:240
#1  0xc0581e1c in boot (howto=16640) at ../../../kern/kern_shutdown.c:374
#2  0xc0582186 in poweroff_wait (junk=0xc074b9b9, howto=2015)
    at ../../../kern/kern_shutdown.c:552
#3  0xc05d70a6 in vput (vp=0xdfcc3be8) at ../../../kern/vfs_subr.c:2015
#4  0xc068712f in ffs_unmount (mp=0xc6332800, mntflags=524288, td=0xc2282540)
    at ../../../ufs/ffs/ffs_vfsops.c:1002
#5  0xc05d2bfe in dounmount (mp=0xc6332800, flags=524288, td=0xc2282540)
    at ../../../kern/vfs_mount.c:1155
#6  0xc05d89df in vfs_unmountall () at ../../../kern/vfs_subr.c:3002
#7  0xc0581cc0 in boot (howto=16384) at ../../../kern/kern_shutdown.c:359
#8  0xc05815ed in reboot (td=0x0, uap=0x0) 
at ../../../kern/kern_shutdown.c:178

Now, just before I rebooted, I had a single ata I/O error during a package 
install:

ad0: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=40<UNCORRECTABLE> 
LBA=15887823

Which resulted in an EIO out in userland. When I was working on the splitfs 
floppy stuff and was running into the 64k boundary bug which gave me 
truncated mfsroots, I would get this same panic when the kernel tried to 
mount the memory filesystem root.  I'm guessing the kernel had the equivalent 
of an EIO during that mount operation.  Robert Watson (cc'd) also just had 
this panic and I know he's been having problems with ata(4) and I/O errors, 
so I think this panic is the result of some bogus handling of I/O errors 
where the vnode is being double vrele()'d somewhere.

-- 
John Baldwin <jhb_at_FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org