Panic on RELENG_6 (in kqueue?)

Had a crash on RELENG_6:

Memory modified after free 0xc1db0800(1020) val=c22af3ac _at_ 0xc1db0b04
panic: Most recently used by kqueue

cpuid = 0
KDB: enter: panic
[thread pid 6939 tid 100087 ]
Stopped at      kdb_enter+0x2b: nop     
db> 
db> where
Tracing pid 6939 tid 100087 td 0xc2190000
kdb_enter(c0854b84) at kdb_enter+0x2b
panic(c086f463,c0851e37,c086f434,c1db0800,3fc) at panic+0x127
mtrash_ctor(c1db0800,400,0,1) at mtrash_ctor+0x4d
uma_zalloc_arg(c105ac60,0,1) at uma_zalloc_arg+0x10f
malloc(400,c08b6920,1,df653bcc,df653bf4) at malloc+0xae
kqueue_expand(c2ddb500,c08b69e0,4,0) at kqueue_expand+0x69
kqueue_register(c2ddb500,df653bf4,c2190000,1,0) at kqueue_register+0x1b8
kern_kevent(c2190000,3,1,1,df653cc8) at kern_kevent+0xc9
kevent(c2190000,df653d04,6,11,296) at kevent+0x55
syscall(3b,3b,bfbf003b,bfbfb6a0,bfbfb688) at syscall+0x22f
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (363, FreeBSD ELF32, kevent), eip = 0x28271d67, esp =
0xbfbfb5fc, ebp = 0xbfbfbcd8 ---

Output from kgdb:

kgdb: kvm_read: invalid address (0xc1bff900)
kgdb: kvm_read: invalid address (0xc1bff780)
kgdb: kvm_read: invalid address (0xc1bff600)
kgdb: kvm_read: invalid address (0xc1bff480)
kgdb: kvm_read: invalid address (0xc1bff300)
kgdb: kvm_read: invalid address (0xc1bff180)
kgdb: kvm_read: invalid address (0xc1bffd80)
kgdb: kvm_read: invalid address (0xc1bffc00)
kgdb: kvm_read: invalid address (0xc1bffa80)
kgdb: kvm_read: invalid address (0xc1bff000)
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".
#0  doadump () at pcpu.h:165
165	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) #0  doadump () at pcpu.h:165
#1  0xc0468e13 in db_fncall (dummy1=0, dummy2=0, dummy3=0, 
    dummy4=0xdf653950 "|9eß,õ|Àh9eßl9eß\220\a")
    at /usr/src/sys/ddb/db_command.c:489
#2  0xc0468c18 in db_command (last_cmdp=0xc0902a84, cmd_table=0x0, 
    aux_cmd_tablep=0xc0880068, aux_cmd_tablep_end=0xc0880084)
    at /usr/src/sys/ddb/db_command.c:349
#3  0xc0468ce0 in db_command_loop () at /usr/src/sys/ddb/db_command.c:455
#4  0xc046a881 in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_main.c:221
#5  0xc0649a9c in kdb_trap (type=3, code=0, tf=0xdf653a94)
    at /usr/src/sys/kern/subr_kdb.c:473
#6  0xc07ec5fc in trap (frame=
      {tf_fs = -547028984, tf_es = -1067188184, tf_ds = -1065025496, tf_edi = -1064897437, tf_esi = 1, tf_ebp = -547013932, tf_isp = -547013952, tf_ebx = -547013888, tf_edx = 0, tf_ecx = -1056755712, tf_eax = 18, tf_trapno = 3, tf_err = 0, tf_eip = -1067149309, tf_cs = 32, tf_eflags = 662, tf_esp = -547013900, tf_ss = -1067246597}) at /usr/src/sys/i386/i386/trap.c:600
#7  0xc07da09a in calltrap () at /usr/src/sys/i386/i386/exception.s:137
#8  0xdf650008 in ?? ()
#9  0xc0640028 in link_elf_load_file (cls=0xc0854b84, 
    filename=0x100 <Address 0x100 out of bounds>, result=0x0)
    at /usr/src/sys/kern/link_elf.c:640
#10 0xc0631bfb in panic (fmt=0x296 <Address 0x296 out of bounds>)
    at /usr/src/sys/kern/kern_shutdown.c:537
#11 0xc0780329 in mtrash_ctor (mem=0xc1db0800, size=0, arg=0x0, flags=1)
    at /usr/src/sys/vm/uma_dbg.c:138
#12 0xc077ec7f in uma_zalloc_arg (zone=0xc105ac60, udata=0x0, flags=1)
    at /usr/src/sys/vm/uma_core.c:1839
#13 0xc06283c2 in malloc (size=1024, mtp=0xc08b6920, flags=1) at uma.h:276
#14 0xc061954d in kqueue_expand (kq=0xc2ddb500, fops=0x0, ident=4, waitok=0)
    at /usr/src/sys/kern/kern_event.c:1047
#15 0xc0618d38 in kqueue_register (kq=0xc2ddb500, kev=0xdf653bf4, 
    td=0xc2190000, waitok=1) at /usr/src/sys/kern/kern_event.c:790
#16 0xc06188a5 in kern_kevent (td=0xc2190000, fd=3, nchanges=1, nevents=1, 
    k_ops=0xdf653cc8, timeout=0xdf653cc0) at /usr/src/sys/kern/kern_event.c:637
#17 0xc0618749 in kevent (td=0xc2190000, uap=0xdf653d04)
    at /usr/src/sys/kern/kern_event.c:571
#18 0xc07ecde7 in syscall (frame=
      {tf_fs = 59, tf_es = 59, tf_ds = -1078001605, tf_edi = -1077954912, tf_esi = -1077954936, tf_ebp = -1077953320, tf_isp = -547013276, tf_ebx = 674182372, tf_edx = 4, tf_ecx = 4, tf_eax = 363, tf_trapno = 0, tf_err = 2, tf_eip = 673652071, tf_cs = 51, tf_eflags = 662, tf_esp = -1077955076, tf_ss = 59})
    at /usr/src/sys/i386/i386/trap.c:985
#19 0xc07da0ef in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:198
#20 0x0000003b in ?? ()
#21 0x0000003b in ?? ()
#22 0xbfbf003b in ?? ()
#23 0xbfbfb6a0 in ?? ()
#24 0xbfbfb688 in ?? ()
#25 0xbfbfbcd8 in ?? ()
#26 0xdf653d64 in ?? ()
#27 0x282f34e4 in ?? ()
#28 0x00000004 in ?? ()
#29 0x00000004 in ?? ()
#30 0x0000016b in ?? ()
#31 0x00000000 in ?? ()
#32 0x00000002 in ?? ()
#33 0x28271d67 in ?? ()
#34 0x00000033 in ?? ()
#35 0x00000296 in ?? ()
#36 0xbfbfb5fc in ?? ()
#37 0x0000003b in ?? ()
#38 0xd0d0d0d0 in ?? ()
#39 0xd0d0d0d0 in ?? ()
#40 0xd0d0d0d0 in ?? ()
#41 0xd0d0d0d0 in ?? ()
#42 0x2726e000 in ?? ()
#43 0xc2197624 in ?? ()
#44 0xc2190000 in ?? ()
#45 0xdf653844 in ?? ()
#46 0xdf65382c in ?? ()
#47 0xc1bfe300 in ?? ()
#48 0xc0641fc3 in sched_switch (td=0xbfbfb688, newtd=0x282f34e4, flags=Cannot access memory at address 0xbfbfbce8
)
    at /usr/src/sys/kern/sched_4bsd.c:973
Previous frame inner to this frame (corrupt stack?)
(kgdb) 

Tim.

-- 
Tim Bishop
http://www.bishnet.net/tim/
PGP Key: 0x5AE7D984