use after free panic ZFS

From: Larry Rosenman <ler_at_lerctr.org>
Date: Mon, 18 May 2015 07:42:47 -0500
found the following panic this am:

borg.lerctr.org dumped core - see /var/crash/vmcore.5

Sun May 17 23:47:48 CDT 2015

FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #40 r283007: Sat May 16 07:23:43 CDT 2015     root_at_borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER  amd64

panic: Most recently used by solaris

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
Memory modified after free 0xfffff808535ea000(120) val=deadc0dd _at_ 0xfffff808535ea050
panic: Most recently used by solaris

cpuid = 5
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100bfb7660
vpanic() at vpanic+0x189/frame 0xfffffe100bfb76e0
panic() at panic+0x43/frame 0xfffffe100bfb7740
mtrash_dtor() at mtrash_dtor/frame 0xfffffe100bfb7760
uma_zalloc_arg() at uma_zalloc_arg+0x4c2/frame 0xfffffe100bfb77d0
malloc() at malloc+0x198/frame 0xfffffe100bfb7820
zfs_range_lock() at zfs_range_lock+0x4a/frame 0xfffffe100bfb7880
zfs_get_data() at zfs_get_data+0x14c/frame 0xfffffe100bfb78f0
zil_commit() at zil_commit+0x94c/frame 0xfffffe100bfb7a10
zfs_freebsd_fsync() at zfs_freebsd_fsync+0xc8/frame 0xfffffe100bfb7a40
VOP_FSYNC_APV() at VOP_FSYNC_APV+0xf7/frame 0xfffffe100bfb7a70
sys_fsync() at sys_fsync+0x173/frame 0xfffffe100bfb7ae0
amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe100bfb7bf0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe100bfb7bf0
--- syscall (95, FreeBSD ELF64, sys_fsync), rip = 0x801eb5daa, rsp = 0x7fffffffd598, rbp = 0x7fffffffd5b0 ---
Uptime: 1d14h25m26s
Dumping 12469 out of 64457 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%

Reading symbols from /boot/kernel/linux.ko.symbols...done.
Loaded symbols for /boot/kernel/linux.ko.symbols
Reading symbols from /boot/kernel/if_lagg.ko.symbols...done.
Loaded symbols for /boot/kernel/if_lagg.ko.symbols
Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done.
Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols
Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done.
Loaded symbols for /boot/kernel/snd_spicds.ko.symbols
Reading symbols from /boot/kernel/coretemp.ko.symbols...done.
Loaded symbols for /boot/kernel/coretemp.ko.symbols
Reading symbols from /boot/kernel/ichsmb.ko.symbols...done.
Loaded symbols for /boot/kernel/ichsmb.ko.symbols
Reading symbols from /boot/kernel/smbus.ko.symbols...done.
Loaded symbols for /boot/kernel/smbus.ko.symbols
Reading symbols from /boot/kernel/ichwd.ko.symbols...done.
Loaded symbols for /boot/kernel/ichwd.ko.symbols
Reading symbols from /boot/kernel/cpuctl.ko.symbols...done.
Loaded symbols for /boot/kernel/cpuctl.ko.symbols
Reading symbols from /boot/kernel/crypto.ko.symbols...done.
Loaded symbols for /boot/kernel/crypto.ko.symbols
Reading symbols from /boot/kernel/cryptodev.ko.symbols...done.
Loaded symbols for /boot/kernel/cryptodev.ko.symbols
Reading symbols from /boot/kernel/dtraceall.ko.symbols...done.
Loaded symbols for /boot/kernel/dtraceall.ko.symbols
Reading symbols from /boot/kernel/profile.ko.symbols...done.
Loaded symbols for /boot/kernel/profile.ko.symbols
Reading symbols from /boot/kernel/dtrace.ko.symbols...done.
Loaded symbols for /boot/kernel/dtrace.ko.symbols
Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done.
Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols
Reading symbols from /boot/kernel/systrace.ko.symbols...done.
Loaded symbols for /boot/kernel/systrace.ko.symbols
Reading symbols from /boot/kernel/sdt.ko.symbols...done.
Loaded symbols for /boot/kernel/sdt.ko.symbols
Reading symbols from /boot/kernel/lockstat.ko.symbols...done.
Loaded symbols for /boot/kernel/lockstat.ko.symbols
Reading symbols from /boot/kernel/fasttrap.ko.symbols...done.
Loaded symbols for /boot/kernel/fasttrap.ko.symbols
Reading symbols from /boot/kernel/fbt.ko.symbols...done.
Loaded symbols for /boot/kernel/fbt.ko.symbols
Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done.
Loaded symbols for /boot/kernel/dtnfscl.ko.symbols
Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done.
Loaded symbols for /boot/kernel/dtmalloc.ko.symbols
Reading symbols from /boot/modules/vboxdrv.ko...done.
Loaded symbols for /boot/modules/vboxdrv.ko
Reading symbols from /boot/modules/nvidia.ko...done.
Loaded symbols for /boot/modules/nvidia.ko
Reading symbols from /boot/kernel/ipmi.ko.symbols...done.
Loaded symbols for /boot/kernel/ipmi.ko.symbols
Reading symbols from /boot/kernel/ipmi_linux.ko.symbols...done.
Loaded symbols for /boot/kernel/ipmi_linux.ko.symbols
Reading symbols from /boot/kernel/radeonkms.ko.symbols...done.
Loaded symbols for /boot/kernel/radeonkms.ko.symbols
Reading symbols from /boot/kernel/iicbb.ko.symbols...done.
Loaded symbols for /boot/kernel/iicbb.ko.symbols
Reading symbols from /boot/kernel/iicbus.ko.symbols...done.
Loaded symbols for /boot/kernel/iicbus.ko.symbols
Reading symbols from /boot/kernel/iic.ko.symbols...done.
Loaded symbols for /boot/kernel/iic.ko.symbols
Reading symbols from /boot/kernel/drm2.ko.symbols...done.
Loaded symbols for /boot/kernel/drm2.ko.symbols
Reading symbols from /boot/kernel/radeonkmsfw_R100_cp.ko.symbols...done.
Loaded symbols for /boot/kernel/radeonkmsfw_R100_cp.ko.symbols
Reading symbols from /boot/kernel/uhid.ko.symbols...done.
Loaded symbols for /boot/kernel/uhid.ko.symbols
Reading symbols from /boot/kernel/ums.ko.symbols...done.
Loaded symbols for /boot/kernel/ums.ko.symbols
Reading symbols from /boot/modules/vboxnetflt.ko...done.
Loaded symbols for /boot/modules/vboxnetflt.ko
Reading symbols from /boot/kernel/netgraph.ko.symbols...done.
Loaded symbols for /boot/kernel/netgraph.ko.symbols
Reading symbols from /boot/kernel/ng_ether.ko.symbols...done.
Loaded symbols for /boot/kernel/ng_ether.ko.symbols
Reading symbols from /boot/modules/vboxnetadp.ko...done.
Loaded symbols for /boot/modules/vboxnetadp.ko
#0  doadump (textdump=Unhandled dwarf expression opcode 0x93
) at pcpu.h:221
221	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) #0  doadump (textdump=Unhandled dwarf expression opcode 0x93
) at pcpu.h:221
#1  0xffffffff80a839b5 in kern_reboot (howto=Unhandled dwarf expression opcode 0x93
)
    at /usr/src/sys/kern/kern_shutdown.c:447
#2  0xffffffff80a83fa8 in vpanic (fmt=<value optimized out>, 
    ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:744
#3  0xffffffff80a83ff3 in panic (fmt=0x0)
    at /usr/src/sys/kern/kern_shutdown.c:675
#4  0xffffffff80d13750 in mtrash_ctor (mem=<value optimized out>, 
    size=<value optimized out>, arg=<value optimized out>, 
    flags=<value optimized out>) at /usr/src/sys/vm/uma_dbg.c:138
#5  0xffffffff80d0f6d2 in uma_zalloc_arg (zone=0xfffff80ffffc9680, udata=0x0, 
    flags=2) at /usr/src/sys/vm/uma_core.c:2197
#6  0xffffffff80a64158 in malloc (size=<value optimized out>, 
    mtp=0xffffffff815e16e0, flags=<value optimized out>) at uma.h:336
#7  0xffffffff80402b4a in zfs_range_lock (zp=0xfffff8075e835730, off=9158656, 
    len=8192, type=Unhandled dwarf expression opcode 0x93
)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_rlock.c:432
#8  0xffffffff8040886c in zfs_get_data (arg=<value optimized out>, 
    lr=<value optimized out>, 
    buf=0xfffffe0662be8178 <Address 0xfffffe0662be8178 out of bounds>, 
    zio=0xfffff80d78b89ac8)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1250
#9  0xffffffff8041c71c in zil_commit (zilog=0xfffff800185c1400, 
    foid=<value optimized out>)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:1108
#10 0xffffffff80410168 in zfs_freebsd_fsync (ap=<value optimized out>)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:2747
#11 0xffffffff80fdfcd7 in VOP_FSYNC_APV (vop=<value optimized out>, 
    a=<value optimized out>) at vnode_if.c:1328
#12 0xffffffff80b40883 in sys_fsync (td=0xfffff8011b253940, 
    uap=<value optimized out>) at vnode_if.h:549
#13 0xffffffff80e968da in amd64_syscall (td=0xfffff8011b253940, traced=0)
    at subr_syscall.c:133
#14 0xffffffff80e767bb in Xfast_syscall ()
    at /usr/src/sys/amd64/amd64/exception.S:395
#15 0x0000000801eb5daa in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language:  auto; currently minimal
(kgdb) 

I have the core. 
-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640                 E-Mail: ler_at_lerctr.org
US Mail: 108 Turvey Cove, Hutto, TX 78634-5688
Received on Mon May 18 2015 - 10:43:01 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:57 UTC