Re: svn commit: r225474 - in head/sys: amd64/amd64 amd64/ia32 i386/i386 ia64/ia32 ia64/ia64 kern powerpc/aim powerpc/booke sparc64/sparc64 sys

On Wed, Sep 14, 2011 at 4:10 PM, Garrett Cooper <yanegomi_at_gmail.com> wrote:
> On Sep 14, 2011, at 2:09 PM, Kostik Belousov wrote:
>
>> [It seems that distribution list can be trimmed without any bad
>> consequences]
>> On Wed, Sep 14, 2011 at 01:50:51PM -0700, Garrett Cooper wrote:
>>> On Sun, Sep 11, 2011 at 9:05 AM, Konstantin Belousov <kib_at_freebsd.org> wrote:
>>>> Author: kib
>>>> Date: Sun Sep 11 16:05:09 2011
>>>> New Revision: 225474
>>>> URL: http://svn.freebsd.org/changeset/base/225474
>>>>
>>>> Log:
>>>>  Inline the syscallenter() and syscallret(). This reduces the time measured
>>>>  by the syscall entry speed microbenchmarks by ~10% on amd64.
>>>>
>>>>  Submitted by: jhb
>>>>  Approved by:  re (bz)
>>>>  MFC after:    2 weeks
>>>
>>>    This change completely breaks ZFS mounting (for some odd reason)
>>> with the following backtrace.
>>>
>>> #0  doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:260
>>> 260     /usr/src/sys/kern/kern_shutdown.c: No such file or directory.
>>>        in /usr/src/sys/kern/kern_shutdown.c
>>> (kgdb) #0  doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:260
>>> #1  0xffffffff802b1cd0 in db_dump (dummy=Variable "dummy" is not available.
>>> )
>>>    at /usr/src/sys/ddb/db_command.c:537
>>> #2  0xffffffff802b12c1 in db_command (last_cmdp=0xffffffff809b96c0,
>>> cmd_table=Variable "cmd_table" is not available.
>>>
>>> ) at /usr/src/sys/ddb/db_command.c:448
>>> #3  0xffffffff802b1510 in db_command_loop ()
>>>    at /usr/src/sys/ddb/db_command.c:501
>>> #4  0xffffffff802b3664 in db_trap (type=Variable "type" is not available.
>>> ) at /usr/src/sys/ddb/db_main.c:229
>>> #5  0xffffffff804b29d1 in kdb_trap (type=3, code=0, tf=0xffffff8231a5f3d0)
>>>    at /usr/src/sys/kern/subr_kdb.c:631
>>> #6  0xffffffff80646ac8 in trap (frame=0xffffff8231a5f3d0)
>>>    at /usr/src/sys/amd64/amd64/trap.c:590
>>> #7  0xffffffff8063113f in calltrap ()
>>>    at /usr/src/sys/amd64/amd64/exception.S:228
>>> #8  0xffffffff804b277b in kdb_enter (why=0xffffffff806e022b "panic",
>>>    msg=0x80 <Address 0x80 out of bounds>) at cpufunc.h:63
>>> #9  0xffffffff8047db5c in panic (fmt=Variable "fmt" is not available.
>>> )
>>>    at /usr/src/sys/kern/kern_shutdown.c:599
>>> #10 0xffffffff8046e5cc in _mtx_assert (m=Variable "m" is not available.
>>> )
>>>    at /usr/src/sys/kern/kern_mutex.c:706
>>> #11 0xffffffff80620f31 in vm_page_free_toq (m=0xfffffe021bf3d1f0)
>>>    at /usr/src/sys/vm/vm_page.c:1756
>>> #12 0xffffffff80c77938 in zfs_freebsd_getpages () from /boot/kernel/zfs.ko
>>> #13 0xffffffff8046ebd6 in _mtx_unlock_flags (m=0xfffffe0006dc7000,
>>>    opts=421100272, file=0xfffffe0006dc70e8 "?P?\200????", line=1)
>>>    at /usr/src/sys/kern/kern_mutex.c:223
>>> Previous frame inner to this frame (corrupt stack?)
>>> (kgdb)
>>
>> The backtrace is impossible and truncated. You probably used unmatched
>> kernel for vmcore, or might be, the zfs.ko is installed without debugging
>> symbols.
>>
>> The change you pointed the finger to has very low probability of causing
>> the panic for vm_page_free_toq(), it is for unrelated part of the kernel.
>> Can you do full rebuild of the kernel without any possible local changes
>> and retest ?
>>
>> If still getting panic, make sure that both kernel and all modules are
>> installed with debug symbols.
>
> zfs.ko wasn't built with symbols because I did make -C /sys/modules/zfs all install; will rebuild my kernel and submit my results again :).

    Weird. The new kernel didn't exhibit the problem after I blew away
/usr/obj (and it contains all of the changes that the old kernel
had).. I guess I'll have to chock this up to improperly compiled
sources.. Sorry for the noise.
-Garrett