Re: GPF on Current kernel

On Mon, Jul 06, 2015 at 06:02:11PM -0500, Larry Rosenman wrote:
>  
> 
> I've enabled DEBUG_REDZONE, and have DEBUG_MEMGUARD in the system, but
> have zero clue as to what to set for memguard :( 
> 
> Could I get some guidance? 
> 
> On 2015-07-06 17:43, K. Macy wrote: 
> 
> > That's really disturbing, but could be an artifact of limitations in debug info fidelity. If curthread is corrupt I would expect things to blow up almost instantly. 
> > 
> > Try enabling memguard for the zone or mtype for the object that has the corrupt mutex field. 
> > 
> > -K 
> > On Jul 6, 2015 5:48 AM, "Larry Rosenman" <ler_at_lerctr.org> wrote:
> > actually, the following are interesting further down:
> > #13 0xffffffff808af473 in sys_write (td=0xdeadc0dedeadc116,
> > uap=<value optimized out>) at /usr/src/sys/kern/sys_generic.c:398
> > #14 0xffffffff80b57a6d in amd64_syscall (td=0xfffff80736fda4c0, traced=0)
> > at subr_syscall.c:133
> > 
> > the td value.
> > 
> > What would you suggest? This has become intermittent :(
> > 
> > On 2015-07-06 00:42, K. Macy wrote:
> > The value of the mutex in the stack trace indicates that it's a use after
> > free. There are various memory debugging options (memguard, redzone) that
> > may help track it down.
> > 
> > -K
> > On Jul 5, 2015 2:37 PM, "Larry Rosenman" <ler_at_lerctr.org> wrote:
> > 
> > I've gotten a couple of these:
> > 
> > borg.lerctr.org [1] dumped core - see /var/crash/vmcore.3
> > 
> > Sun Jul 5 16:32:12 CDT 2015
> > 
> > FreeBSD borg.lerctr.org [1] 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r285159: Sun
> > Jul 5 09:35:33 CDT 2015 root_at_borg.lerctr.org:/usr/obj/usr/src/sys/LER
> > amd64
> > 
> > panic: general protection fault
> > 
> > GNU gdb 6.1.1 [FreeBSD]
> > Copyright 2004 Free Software Foundation, Inc.
> > GDB is free software, covered by the GNU General Public License, and you
> > are
> > welcome to change it and/or distribute copies of it under certain
> > conditions.
> > Type "show copying" to see the conditions.
> > There is absolutely no warranty for GDB. Type "show warranty" for details.
> > This GDB was configured as "amd64-marcel-freebsd"...
> > 
> > Unread portion of the kernel message buffer:
> > 
> > Fatal trap 9: general protection fault while in kernel mode
> > cpuid = 0; apic id = 00
> > instruction pointer = 0x20:0xffffffff8083a15f
> > stack pointer = 0x28:0xfffffe100ca1d8a0
> > frame pointer = 0x28:0xfffffe100ca1d8e0
> > code segment = base 0x0, limit 0xfffff, type 0x1b
> > = DPL 0, pres 1, long 1, def32 0, gran 1
> > processor eflags = interrupt enabled, resume, IOPL = 0
> > current process = 73971 (make)
> > trap number = 9
> > panic: general protection fault
> > cpuid = 0
> > KDB: stack backtrace:
> > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> > 0xfffffe100ca1d490
> > vpanic() at vpanic+0x189/frame 0xfffffe100ca1d510
> > panic() at panic+0x43/frame 0xfffffe100ca1d570
> > trap_fatal() at trap_fatal+0x355/frame 0xfffffe100ca1d5d0
> > trap() at trap+0x813/frame 0xfffffe100ca1d7e0
> > calltrap() at calltrap+0x8/frame 0xfffffe100ca1d7e0
> > --- trap 0x9, rip = 0xffffffff8083a15f, rsp = 0xfffffe100ca1d8a0, rbp =
> > 0xfffffe100ca1d8e0 ---
> > __mtx_lock_flags() at __mtx_lock_flags+0x4f/frame 0xfffffe100ca1d8e0
> > doselwakeup() at doselwakeup+0xb5/frame 0xfffffe100ca1d920
> > pipeselwakeup() at pipeselwakeup+0x46/frame 0xfffffe100ca1d940
> > pipe_write() at pipe_write+0x13d2/frame 0xfffffe100ca1d9f0
> > dofilewrite() at dofilewrite+0x8a/frame 0xfffffe100ca1da40
> > kern_writev() at kern_writev+0x68/frame 0xfffffe100ca1da90
> > sys_write() at sys_write+0x63/frame 0xfffffe100ca1dae0
> > amd64_syscall() at amd64_syscall+0x25d/frame 0xfffffe100ca1dbf0
> > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe100ca1dbf0
> > --- syscall (4, FreeBSD ELF64, sys_write), rip = 0x45ae8a, rsp =
> > 0x7fffffffd328, rbp = 0x7fffffffd350 ---
> > Uptime: 6h39m51s
> > Dumping 3186 out of 64460
> > MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
> > 
> > Reading symbols from /boot/kernel/zfs.ko.symbols...done.
> > Loaded symbols for /boot/kernel/zfs.ko.symbols
> > Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
> > Loaded symbols for /boot/kernel/opensolaris.ko.symbols
> > Reading symbols from /boot/kernel/linux.ko.symbols...done.
> > Loaded symbols for /boot/kernel/linux.ko.symbols
> > Reading symbols from /boot/kernel/linux_common.ko.symbols...done.
> > Loaded symbols for /boot/kernel/linux_common.ko.symbols
> > Reading symbols from /boot/kernel/if_lagg.ko.symbols...done.
> > Loaded symbols for /boot/kernel/if_lagg.ko.symbols
> > Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done.
> > Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols
> > Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done.
> > Loaded symbols for /boot/kernel/snd_spicds.ko.symbols
> > Reading symbols from /boot/kernel/coretemp.ko.symbols...done.
> > Loaded symbols for /boot/kernel/coretemp.ko.symbols
> > Reading symbols from /boot/kernel/ichsmb.ko.symbols...done.
> > Loaded symbols for /boot/kernel/ichsmb.ko.symbols
> > Reading symbols from /boot/kernel/smbus.ko.symbols...done.
> > Loaded symbols for /boot/kernel/smbus.ko.symbols
> > Reading symbols from /boot/kernel/ichwd.ko.symbols...done.
> > Loaded symbols for /boot/kernel/ichwd.ko.symbols
> > Reading symbols from /boot/kernel/cpuctl.ko.symbols...done.
> > Loaded symbols for /boot/kernel/cpuctl.ko.symbols
> > Reading symbols from /boot/kernel/cryptodev.ko.symbols...done.
> > Loaded symbols for /boot/kernel/cryptodev.ko.symbols
> > Reading symbols from /boot/kernel/dtraceall.ko.symbols...done.
> > Loaded symbols for /boot/kernel/dtraceall.ko.symbols
> > Reading symbols from /boot/kernel/profile.ko.symbols...done.
> > Loaded symbols for /boot/kernel/profile.ko.symbols
> > Reading symbols from /boot/kernel/dtrace.ko.symbols...done.
> > Loaded symbols for /boot/kernel/dtrace.ko.symbols
> > Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done.
> > Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols
> > Reading symbols from /boot/kernel/systrace.ko.symbols...done.
> > Loaded symbols for /boot/kernel/systrace.ko.symbols
> > Reading symbols from /boot/kernel/sdt.ko.symbols...done.
> > Loaded symbols for /boot/kernel/sdt.ko.symbols
> > Reading symbols from /boot/kernel/lockstat.ko.symbols...done.
> > Loaded symbols for /boot/kernel/lockstat.ko.symbols
> > Reading symbols from /boot/kernel/fasttrap.ko.symbols...done.
> > Loaded symbols for /boot/kernel/fasttrap.ko.symbols
> > Reading symbols from /boot/kernel/fbt.ko.symbols...done.
> > Loaded symbols for /boot/kernel/fbt.ko.symbols
> > Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done.
> > Loaded symbols for /boot/kernel/dtnfscl.ko.symbols
> > Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done.
> > Loaded symbols for /boot/kernel/dtmalloc.ko.symbols
> > Reading symbols from /boot/modules/vboxdrv.ko...done.
> > Loaded symbols for /boot/modules/vboxdrv.ko
> > Reading symbols from /boot/modules/nvidia.ko...done.
> > Loaded symbols for /boot/modules/nvidia.ko
> > Reading symbols from /boot/kernel/ipmi.ko.symbols...done.
> > Loaded symbols for /boot/kernel/ipmi.ko.symbols
> > Reading symbols from /boot/kernel/ipmi_linux.ko.symbols...done.
> > Loaded symbols for /boot/kernel/ipmi_linux.ko.symbols
> > Reading symbols from /boot/kernel/radeonkms.ko.symbols...done.
> > Loaded symbols for /boot/kernel/radeonkms.ko.symbols
> > Reading symbols from /boot/kernel/iicbb.ko.symbols...done.
> > Loaded symbols for /boot/kernel/iicbb.ko.symbols
> > Reading symbols from /boot/kernel/iicbus.ko.symbols...done.
> > Loaded symbols for /boot/kernel/iicbus.ko.symbols
> > Reading symbols from /boot/kernel/iic.ko.symbols...done.
> > Loaded symbols for /boot/kernel/iic.ko.symbols
> > Reading symbols from /boot/kernel/drm2.ko.symbols...done.
> > Loaded symbols for /boot/kernel/drm2.ko.symbols
> > Reading symbols from /boot/kernel/radeonkmsfw_R100_cp.ko.symbols...done.
> > Loaded symbols for /boot/kernel/radeonkmsfw_R100_cp.ko.symbols
> > Reading symbols from /boot/kernel/uhid.ko.symbols...done.
> > Loaded symbols for /boot/kernel/uhid.ko.symbols
> > Reading symbols from /boot/modules/vboxnetflt.ko...done.
> > Loaded symbols for /boot/modules/vboxnetflt.ko
> > Reading symbols from /boot/kernel/netgraph.ko.symbols...done.
> > Loaded symbols for /boot/kernel/netgraph.ko.symbols
> > Reading symbols from /boot/kernel/ng_ether.ko.symbols...done.
> > Loaded symbols for /boot/kernel/ng_ether.ko.symbols
> > Reading symbols from /boot/modules/vboxnetadp.ko...done.
> > Loaded symbols for /boot/modules/vboxnetadp.ko

First reproduce the issue without the vbox modules loaded.