Re: Fatal trap 3 in vnlru() and lock order problem in netinet/raw_ip.c:199

From: J.R. Oldroyd <fbsd_at_opal.com>
Date: Wed, 2 Mar 2005 11:05:09 -0500
I suspect that setting debug.mpsafevfs=0 in loader.conf won't help because
sysctl shows that it is already 0.

Kernel is -current of 2/22 with atamk3l, generic config + IPFIREWALL,
IPDIVERT and IP6FWIREWALL, debug is enabled.

The freeze, it happened again.  About 48 hours uptime, then it froze
again.  This time, I was away from my desk so the screensaver had
kicked in and blanked the screen.  When I returned and wiggled the
mouse, it did not come back.  Neither did it drop into the debugger,
or if it did, I was last in X, so I could not see the debugger.
Looking at the syslog after the reboot, there is a little more info:

Mar  2 10:01:58 linwhf kernel: cess		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097bb7c
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097bb7c
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097bab8
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097bab8
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b9f4
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b9f4
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b930
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b930
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b86c
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b86c
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b7a8
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b7a8
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b6e4
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b6e4
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b620
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b620
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b55c
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b55c
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b498
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b498
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b3d4
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b3d4
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b310
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b310
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b24c
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b24c
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b188
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b188
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b0c4
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b0c4
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097b000
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097b000
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097af3c
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097af3c
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)
Mar  2 10:01:58 linwhf kernel: trap number		= 3
Mar  2 10:01:58 linwhf kernel: panic: breakpoint instruction fault
Mar  2 10:01:58 linwhf kernel: cpuid = 0
Mar  2 10:01:58 linwhf kernel: KDB: enter: panic
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: 
Mar  2 10:01:58 linwhf kernel: Fatal trap 3: breakpoint instruction fault while in kernel mode
Mar  2 10:01:58 linwhf kernel: cpuid = 0; apic id = 00
Mar  2 10:01:58 linwhf kernel: instruction pointer	= 0x8:0xc0635163
Mar  2 10:01:58 linwhf kernel: stack pointer	        = 0x10:0xc097ae78
Mar  2 10:01:58 linwhf kernel: frame pointer	        = 0x10:0xc097ae78
Mar  2 10:01:58 linwhf kernel: code segment		= base 0x0, limit 0xfffff, type 0x1b
Mar  2 10:01:58 linwhf kernel: = DPL 0, pres 1, def32 1, gran 1
Mar  2 10:01:58 linwhf kernel: processor eflags	= IOPL = 0
Mar  2 10:01:58 linwhf kernel: current process		= 56 (vnlru)

... at this point I hit the button to reboot...

Mar  2 10:01:58 linwhf kernel: Copyright (c) 1992-2005 The FreeBSD Project.
Mar  2 10:01:58 linwhf kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Mar  2 10:01:58 linwhf kernel: The Regents of the University of California. All rights reserved.
Mar  2 10:01:58 linwhf kernel: FreeBSD 6.0-CURRENT #75: Sun Feb 27 18:25:58 EST 2005

... all the normal boot messages follow, then as it goes multiuser...

Mar  2 10:01:58 linwhf kernel: ATA PseudoRAID loaded
Mar  2 10:01:58 linwhf kernel: SMP: AP CPU #1 Launched!
Mar  2 10:01:58 linwhf kernel: Trying to mount root from ufs:/dev/ad0s1a
Mar  2 10:01:58 linwhf kernel: WARNING: / was not properly dismounted
Mar  2 10:01:58 linwhf kernel: WARNING: /tmp was not properly dismounted
Mar  2 10:01:58 linwhf kernel: WARNING: /var/mail was not properly dismounted
Mar  2 10:01:58 linwhf kernel: WARNING: /var/spool was not properly dismounted
Mar  2 10:01:58 linwhf kernel: WARNING: /var/www was not properly dismounted
Mar  2 10:01:58 linwhf kernel: WARNING: /usr was not properly dismounted
Mar  2 10:01:58 linwhf kernel: WARNING: /home was not properly dismounted
Mar  2 10:01:58 linwhf savecore: no dumps found
Mar  2 10:01:58 linwhf named[372]: starting BIND 9.3.0 -u bind
Mar  2 10:01:59 linwhf named[372]: command channel listening on ::1#953
Mar  2 10:01:59 linwhf named[372]: command channel listening on 127.0.0.1#953
Mar  2 10:02:01 linwhf ntpd[484]: ntpd 4.2.0-a Tue Feb 22 16:26:34 EST 2005 (1)
Mar  2 10:02:32 linwhf kernel: lock order reversal
Mar  2 10:02:32 linwhf kernel: 1st 0xc3b65c84 inp (rawinp) _at_ netinet/raw_ip.c:199
Mar  2 10:02:32 linwhf kernel: 2nd 0xc3b64f54 inp (raw6inp) _at_ netinet/raw_ip.c:199
Mar  2 10:02:32 linwhf kernel: KDB: stack backtrace:
Mar  2 10:02:32 linwhf kernel: kdb_backtrace(0,ffffffff,c090ea50,c090e9d8,c089a888) at kdb_backtrace+0x29
Mar  2 10:02:32 linwhf kernel: witness_checkorder(c3b64f54,9,c084615c,c7) at witness_checkorder+0x54c
Mar  2 10:02:32 linwhf kernel: _mtx_lock_flags(c3b64f54,0,c0846153,c7) at _mtx_lock_flags+0x5b
Mar  2 10:02:32 linwhf kernel: rip_input(c3b56500,14,e4f70c48,b,14) at rip_input+0x64
Mar  2 10:02:32 linwhf kernel: icmp_input(c3b56500,14,100007f,0,0) at icmp_input+0x4ed
Mar  2 10:02:32 linwhf kernel: ip_input(c3b56500) at ip_input+0x50d
Mar  2 10:02:32 linwhf kernel: netisr_processqueue(c094e058) at netisr_processqueue+0x6e
Mar  2 10:02:32 linwhf kernel: swi_net(0) at swi_net+0xbe
Mar  2 10:02:32 linwhf kernel: ithread_loop(c34da180,e4f70d48,c34da180,c060b3f0,0) at ithread_loop+0x120
Mar  2 10:02:32 linwhf kernel: fork_exit(c060b3f0,c34da180,e4f70d48) at fork_exit+0xa4
Mar  2 10:02:32 linwhf kernel: fork_trampoline() at fork_trampoline+0x8
Mar  2 10:02:32 linwhf kernel: --- trap 0x1, eip = 0, esp = 0xe4f70d7c, ebp = 0 ---
Mar  2 10:03:26 linwhf fsck: /dev/ad0s1d: UNREF FILE I=113  OWNER=jr MODE=100600
Mar  2 10:03:26 linwhf fsck: /dev/ad0s1d: SIZE=1190 MTIME=Mar  1 22:34 2005  (CLEARED)
Mar  2 10:03:26 linwhf fsck: /dev/ad0s1d: UNREF FILE I=118  OWNER=root MODE=100600
Mar  2 10:03:26 linwhf fsck: /dev/ad0s1d: SIZE=0 MTIME=Mar  2 03:01 2005  (CLEARED)
Mar  2 10:03:26 linwhf fsck: /dev/ad0s1d: UNREF FILE I=120  OWNER=root MODE=100600
Mar  2 10:03:26 linwhf fsck: /dev/ad0s1d: SIZE=0 MTIME=Mar  2 03:21 2005  (CLEARED)
Mar  2 10:03:26 linwhf fsck: /dev/ad0s1d: Reclaimed: 0 directories, 3 files, 0 fragments
Mar  2 10:03:26 linwhf fsck: /dev/ad0s1d: 255 files, 2965 used, 123874 free (114 frags, 15470 blocks, 0.1% fragmentation)
Mar  2 10:03:27 linwhf fsck: /dev/ad0s1f: 9 files, 194 used, 126645 free (21 frags, 15828 blocks, 0.0% fragmentation)
Mar  2 10:03:27 linwhf fsck: /dev/ad0s1g: CANNOT ADJUST NUMBER OF FREE INODES: 2
Mar  2 10:03:27 linwhf fsck: /dev/ad0s1g: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY.

The subsequent manual fsck unearthed significant trashing of a
hierarchy of files I was using yesterday.  There are now 1764 entries
in lost+found!  Random sampling suggests that these appear to be
files that are part of a large scientific library which I was
compiling about 24 hours ago.

I note from going back through my syslog that this freeze is always
in vnlru; I have 2 GB of mem, and this library I was compiling does
consist of lots and lots of files.


There may also be a second problem.  Note the "lock order reversal"
backtrace above showing rawinp and raw6inp in netinet/raw_ip.c:199.
This happened both during that boot and on the reboot following
the full fsck.

	-jr





On Feb 28, 16:14, Robert Watson wrote:
> On Mon, 28 Feb 2005, J.R. Oldroyd wrote:
> 
> > I am seeing similar problem, using -current from 2/22 with atamk3l.
> > Running on hyperthreaded i386 here.
> > <snip>
> > I have encountered this crash about five or six times since 2/22,
> > but I've been otherwise occupied and not able to look at it.  Crash
> > is infrequent, as in system's OK for many hours, then just freezes.
> > Need to press the button to get it back.
> > 
> > Config is generic with IPFIREWALL, IPDIVERT and IPV6FIREWALL added
> > and debugging turned off.
> > 
> > Since I have no other info at this time, I just built and installed a
> > kernel with debugging on, so that next time I may have more details. 
> > Don't yet have any repeatable way to reproduce this. 
> 
> If you set "debug.mpsafevfs=0" in loader.conf and reboot, does it go away?
> 
> Robert N M Watson
> 
> 
> > 
> > Previous kernel, 1/31 without atamk3, was stable.
> > 
> > 	-jr
> > 
> > 
> > 
> > On Feb 25, 21:21, Kris Kennaway wrote:
> > > On Fri, Feb 25, 2005 at 04:47:14PM -0800, Doug White wrote:
> > > > On Thu, 24 Feb 2005, Kris Kennaway wrote:
> > > > 
> > > > > > fault virtual address   = 0x7562676b
> > > > 
> > > > I agree with Dan, this is bogus. kernload() is the offset from kernbase
> > > > where the ELF headers get stuck. I suspect ddb is resolving it like it
> > > > resolves end -- its beyond the beginning of the kernel so it picks the
> > > > next best match, like end shows up beyond the end of the symbol table.
> > > > (FYI end usually indicates calls into a KLD.)
> > > > 
> > > > > > current process         = 52613 (getty)
> > > > > > Tracing pid 52613 tid 100360 td 0xd2d3a000
> > > > > > kernload(cd533500,3,2000,d2d3a000,3) at 0x7562676b
> > > > > > devfs_open(f8225a4c,c072025a,1e6,c07205ff,d235f134) at devfs_open+0x291
> > > > 
> > > > Can you get an addr2line on this devfs_open call?  It appears to have
> > > > tried to open an incompletely initialized tty device.  Which one would be
> > > > nice to know :-)
> > > 
> > > It was ttyd0.  phk couldn't figure out how this happened, so I'm
> > > trying to recreate it.
> > > 
> > > Kris
> > > 
> > > > > > VOP_OPEN_APV(c07340a0,f8225a4c,3,c076d398,1) at VOP_OPEN_APV+0x9e
> > > > > > vn_open_cred(f8225bbc,f8225cbc,860,cd33e180,1) at vn_open_cred+0x45b
> > > > > > vn_open(f8225bbc,f8225cbc,860,1,d2d3a000) at vn_open+0x33
> > > > > > kern_open(d2d3a000,804f860,0,3,804f860) at kern_open+0xca
> > > > > > open(d2d3a000,f8225d14,3a6,c071c691,d2d3a000) at open+0x36
> > > > > > syscall(2f,2f,2f,2,804f860) at syscall+0x2c4
> > > > > > Xint0x80_syscall() at Xint0x80_syscall+0x1f
> > > > > > --- syscall (5, FreeBSD ELF32, open), eip = 0x280ca2cb, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
> > > > > > db>
> > > > >
> > > > > Kris
> > > > >
> > > > 
> > > > -- 
> > > > Doug White                    |  FreeBSD: The Power to Serve
> > > > dwhite_at_gumbysoft.com          |  www.FreeBSD.org
> > > > _______________________________________________
> > > > freebsd-current_at_freebsd.org mailing list
> > > > http://lists.freebsd.org/mailman/listinfo/freebsd-current
> > > > To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
> > > > 
> > 
> > 
> > 
Received on Wed Mar 02 2005 - 15:05:17 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:38:29 UTC