Re: 7.0 RC2 kernel panic with Kqemu/AMD64

On Sun, Feb 17, 2008 at 06:51:18AM -0600, John Marino wrote:
> Hello Juergen,
> I decided to build a debug kernel as you suggested with DDB, KDB,
> KDB_TRACE, and KDB_UNATTENDED as you suggested.  The first time the kernel
> panicked, I did not get a dump, but the system did save the second panic. 
> The backtrace is about the same, but the initial information looks much
> better.
> 
> I hope this helps,
> John
> 
> 
> draco-root# kgdb kernel.debug /usr/local/crash/vmcore.1
> [GDB will not be able to debug user-mode threads:
> /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "amd64-marcel-freebsd".
> 
> Unread portion of the kernel message buffer:
> kernel
> 
> tFraatpa l trap 12 wi1t2h:  interpraugpet sf aduilsta bwlheidle
>  i
> n
>  Fkaetranle lt rmaopd e1
> :c pupirdi v=i l1e;g eadp iicn sitdr u=c tion0 1f
> afualutl tw hviilret uianl  kaedrdnreels sm	o=d e0
> xc0p
> ufiadu l=t  0;c oadpei	c	 =i ds u=p e0r0v
> iisnosrt rruecatdi odna tpao,i nptaegre	 =n o0tx 2pfr3e0s:en0tx
> fifnfsftfrfufcft8i0o8n3 bpbocidn
> tsetra	c=k  0pxo8i:n0txer	 f f f f f f f f=8 004x710b:a01x5f
> fsftfafcfk fpfoaibn9tde6r6	e 0
>  f r a m e  =p o0ixn1t0e:r0	x    f f f f f=f f0fxa1b09:d06xbcf0f
> fffrfafmfef 8p0o9i8n1tceer0
>  c o d e   s e g=m e0nxt1		0=: 0bxase f0fxf0f,f flfifmaibt9 d06xc00,0
> tcyopdee  0sxe0gm
> e	nt				==  DbPaLs e0 ,0 xp0r,e sl i0m,i tl o0nxg f0f,f fdfe,f32 0, gran 0
> kernel trap 12 with interrupts disabled

OK looks like indeed both cpus are crashing, maybe try setting
PRINTF_BUFR_SIZE as others have suggested.
> 
> 
> Fatal trap 12: page fault while in kernel mode
> cpuid = 0; apic id = 00
> fault virtual address	= 0xffffffffffffffd3
> fault code		= supervisor write data, page not present
> instruction pointer	= 0x8:0xffffff00010e1680

 OK I doubt thats inside the kernel, but you could try doing
`i li *0xffffff00010e1680' in kgdb to make sure...

> stack pointer	        = 0x10:0xffffffffab9d64e0
> frame pointer	        = 0x10:0xffffffff80a93640
> code segment		= base 0x0, limit 0xfffff, type 0x1b
> 			= DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags	= resume, IOPL = 0
> current process		= 999 (qemu-system-x86_64)
> trap number		= 12
> panic: page fault
> cpuid = 0
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
> panic() at panic+0x17a
> trap_fatal() at trap_fatal+0x29f
> trap() at trap+0x242
> calltrap() at calltrap+0x8
> --- trap 0xc, rip = 0xffffff00010e1680, rsp = 0xffffffffab9d64e0, rbp =
> 0xffffffff80a93640 ---
> dmapbase() at 0xffffff00010e1680
> Uptime: 29m47s
> Dumping 1983 MB (2 chunks)
>   chunk 0: 1MB (156 pages) ... ok
>   chunk 1: 1983MB (507568 pages) 1967 1951 1935 1919 1903 1887 1871 1855
> 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695 1679 1663 1647 1631
> 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471 1455 1439 1423 1407
> 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247 1231 1215 1199 1183
> 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023 1007 991 975 959 943
> 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655
> 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367
> 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63
> 47 31 15
> 
> #0  doadump () at pcpu.h:194
> 194		__asm __volatile("movq %%gs:0,%0" : "=r" (td));
> 
> 
> (kgdb) backtrace
> #0  doadump () at pcpu.h:194
> #1  0xffffffff80486dd8 in boot (howto=260) at
> /usr/src/sys/kern/kern_shutdown.c:409
> #2  0xffffffff80487237 in panic (fmt=Variable "fmt" is not available.) at
> /usr/src/sys/kern/kern_shutdown.c:563
> #3  0xffffffff8074857f in trap_fatal (frame=0xc, eva=Variable "eva" is not
> available.) at /usr/src/sys/amd64/amd64/trap.c:724
> #4  0xffffffff80749272 in trap (frame=0xffffffffab9d6430) at
> /usr/src/sys/amd64/amd64/trap.c:251
> #5  0xffffffff8072e60e in calltrap () at
> /usr/src/sys/amd64/amd64/exception.S:169
> #6  0xffffffff8047ba90 in _thread_lock_flags (td=0xffffffffab9d66e0,
> opts=Variable "opts" is not available.) at
> /usr/src/sys/kern/kern_mutex.c:523

 So thats how the backtrace ended, next line was the kdgb prompt?

 Anyway I'm still not enlightened yet what the actual problem might be...
	Juergen