Re: zfs: Fatal trap 12: page fault while in kernel mode

On Jul 29, 2009, at 15:46, Andriy Gapon wrote:

> on 29/07/2009 16:24 Thomas Backman said the following:
>> Hmm, you are indeed right, it's not the same panic. The backtrace I  
>> got
>> just now with INVARIANTS is the one you quoted above.
>> I still get the "_sx_xlock (sx=Variable "sx" is not available.)" and
>> "_sx_xlock_hard (sx=0xffffff00090d5018, ..., opts=Variable "opts"  
>> is not
>> available.)" though.
>> Am I missing some option (I've got GENERIC, minus WITNESS plus  
>> DTRACE,
>> now that INVARIANTS is back in place), or does this "just happen"?
>
> Not sure what this question is about. What option, what "this" :-)
>
>> Here's the "full" backtrace (minus the panic(), trap() etc.):
>>
>> #10 0xffffffff8057dfe7 in calltrap ()
>>    at /usr/src/sys/amd64/amd64/exception.S:224
>> #11 0xffffffff80342b99 in _sx_xlock_hard (sx=0xffffff00090d5018,
>>    tid=18446742974952890368, opts=Variable "opts" is not available.
>> ) at /usr/src/sys/kern/kern_sx.c:575
>> #12 0xffffffff8034350e in _sx_xlock (sx=Variable "sx" is not  
>> available.
>> ) at sx.h:155
>> #13 0xffffffff80af7596 in zfs_freebsd_reclaim () from /boot/kernel/ 
>> zfs.ko
>> #14 0xffffffff805c5c2a in VOP_RECLAIM_APV (vop=0xffffff00090d5018,
>>    a=0xffffff00090d5000) at vnode_if.c:1926
>> #15 0xffffffff803c839e in vgonel (vp=0xffffff0009252588) at  
>> vnode_if.h:830
>> #16 0xffffffff803cc958 in vflush (mp=0xffffff0002cd7bc0, rootrefs=0,
>> flags=0,
>>    td=0xffffff002cffe000) at /usr/src/sys/kern/vfs_subr.c:2449
>> #17 0xffffffff80af2038 in zfs_umount () from /boot/kernel/zfs.ko
>> #18 0xffffffff803c55ca in dounmount (mp=0xffffff0002cd7bc0,  
>> flags=47020992,
>>    td=Variable "td" is not available.
>> ) at /usr/src/sys/kern/vfs_mount.c:1289
>> #19 0xffffffff803c5df8 in unmount (td=0xffffff002cffe000,
>>    uap=0xffffff803e98bbf0) at /usr/src/sys/kern/vfs_mount.c:1174
>> #20 0xffffffff805980bf in syscall (frame=0xffffff803e98bc80)
>>    at /usr/src/sys/amd64/amd64/trap.c:984
>> #21 0xffffffff8057e2c1 in Xfast_syscall ()
>>    at /usr/src/sys/amd64/amd64/exception.S:373
>> #22 0x000000080104e9ec in ?? ()
>> Previous frame inner to this frame (corrupt stack?)
>
> Looks like your zfs module is built without debugging symbols?
> Maybe because was it built/rebuilt individually, not as part of  
> kernel build?
>
> It would be useful to get line number in frame 13 and examine sx  
> object in frame
> 11, esp. sx_lock field.
>
> -- 
> Andriy Gapon
The "this" (above) was referring to variable values not being  
available in a vmcore. :)

The zfs module appears to be built with symbols, and the symbols  
appear to be loaded in kgdb:

Reading symbols from /boot/kernel/zfs.ko...Reading symbols from / 
bootdir/boot/kernel/zfs.ko.symbols...done. done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols  
from /bootdir/boot/kernel/opensolaris.ko.symbols...done. done.
Loaded symbols for /boot/kernel/opensolaris.ko

I didn't build the module(s) individually, either; in the previous  
cases, it was a clean buildworld/buildkernel (even with rm -rf /usr/ 
obj/* beforehand), and in this case "just" a buildkernel (no manual  
cleaning, but no -DNO_CLEAN either).

Regards,
Thomas