Re: Fatal trap 12: page fault while in kernel mode

>>>>> In <863ah2qivi.fsf_at_ra333.heimat.gr.jp> 
>>>>>	NAKAJI Hiroyuki <nakaji_at_jp.freebsd.org> wrote:

> So, what I have to do are three:

> 1. Full upgrade to the latest kernel and userland (world)
> 2. Observe whether a panic occurs, and
> 3. When panic, save the crash dump and get full bt with kgdb

> I hope it will not reach to the step three. Thanks.

Unfortunately, I faced to "db> " on my serial console.

db> bt
Tracing pid 964 tid 100131 td 0xc49876c0
svc_run_internal(e6861d24,c0850793,c44f2780,e6861d38,1,...) at svc_run_internal+
0x575
svc_thread_start(c44f2780,e6861d38,1,0,4836bd14,...) at svc_thread_start+0x10
fork_exit(c0a57fe0,c44f2780,e6861d38) at fork_exit+0x93
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0xc, esp = 0x33, ebp = 0 ---
db> call doadump
Physical memory: 1010 MB
Dumping 250 MB: 235 219 203 187 171 155 139 123 107 91 75 59 43 27 11
Dump complete
= 0xf
db> bt
Tracing pid 964 tid 100131 td 0xc49876c0
svc_run_internal(e6861d24,c0850793,c44f2780,e6861d38,1,...) at svc_run_internal+
0x575
svc_thread_start(c44f2780,e6861d38,1,0,4836bd14,...) at svc_thread_start+0x10
fork_exit(c0a57fe0,c44f2780,e6861d38) at fork_exit+0x93
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0xc, esp = 0x33, ebp = 0 ---
db> reset

And then, I used kgdb.

# kgdb /boot/kernel/kernel.symbols vmcore.9 
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x0
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc0a57a75
stack pointer           = 0x28:0xe6861c44
frame pointer           = 0x28:0xe6861cec
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 964 (nfsd: service)
Physical memory: 1010 MB
Dumping 250 MB: 235 219 203 187 171 155 139 123 107 91 75 59 43 27 11

Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /boot/kernel/linprocfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/linprocfs.ko
Reading symbols from /boot/kernel/linux.ko...Reading symbols from /boot/kernel/linux.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/linux.ko
Reading symbols from /boot/kernel/smbus.ko...Reading symbols from /boot/kernel/smbus.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/smbus.ko
Reading symbols from /boot/kernel/aio.ko...Reading symbols from /boot/kernel/aio.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/aio.ko
Reading symbols from /boot/kernel/mga.ko...Reading symbols from /boot/kernel/mga.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/mga.ko
Reading symbols from /boot/kernel/drm.ko...Reading symbols from /boot/kernel/drm.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/drm.ko
Reading symbols from /boot/kernel/atapicam.ko...Reading symbols from /boot/kernel/atapicam.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/atapicam.ko
Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /boot/kernel/nullfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/nullfs.ko
Reading symbols from /boot/kernel/logo_saver.ko...Reading symbols from /boot/kernel/logo_saver.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/logo_saver.ko
#0  doadump () at pcpu.h:246
246     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) bt
#0  doadump () at pcpu.h:246
#1  0xc04c2a69 in db_fncall (dummy1=-1059812576, dummy2=0, dummy3=-1067505768, 
    dummy4=0xe68619d8 "�:i�\2008'�") at /usr/src/sys/ddb/db_command.c:548
#2  0xc04c2e61 in db_command (last_cmdp=0xc0d3e85c, cmd_table=0x0, dopager=1)
    at /usr/src/sys/ddb/db_command.c:445
#3  0xc04c2fba in db_command_loop () at /usr/src/sys/ddb/db_command.c:498
#4  0xc04c4dfd in db_trap (type=12, code=0) at /usr/src/sys/ddb/db_main.c:229
#5  0xc08a2456 in kdb_trap (type=12, code=0, tf=0xe6861c04)
    at /usr/src/sys/kern/subr_kdb.c:534
#6  0xc0b8059f in trap_fatal (frame=0xe6861c04, eva=0)
    at /usr/src/sys/i386/i386/trap.c:920
#7  0xc0b80860 in trap_pfault (frame=0xe6861c04, usermode=0, eva=0)
    at /usr/src/sys/i386/i386/trap.c:842
#8  0xc0b8126a in trap (frame=0xe6861c04) at /usr/src/sys/i386/i386/trap.c:522
#9  0xc0b652cb in calltrap () at /usr/src/sys/i386/i386/exception.s:165
#10 0xc0a57a75 in svc_run_internal (pool=0xc44f2780, ismaster=0)
    at /usr/src/sys/rpc/svc.c:787
#11 0xc0a57ff0 in svc_thread_start (arg=0xc44f2780)
    at /usr/src/sys/rpc/svc.c:1188
#12 0xc0850793 in fork_exit (callout=0xc0a57fe0 <svc_thread_start>, 
    arg=0xc44f2780, frame=0xe6861d38) at /usr/src/sys/kern/kern_fork.c:821
#13 0xc0b65340 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:270
(kgdb) 

In addition,

# addr2line -e /boot/kernel/kernel.symbols 0xc0a57a75
/usr/src/sys/rpc/svc.c:787

Any help is appreciated. Thanks.
-- 
NAKAJI Hiroyuki