Re: diskless/unionfs panics

On Fri, 22 Apr 2005, Danny Braniss wrote:

> hi,
> 	after much debugging, it seems that the main problem with unionfs is
> that if it's called early in the boot process it will panic the kernel:
>
> trap 12: page fault while in kernel mode
> cpuid = 0; apic id = 00
> fault virtual address   = 0x0
> fault code              = supervisor read, page not present
> instruction pointer     = 0x8:0xffffffff8038e3f5
> stack pointer           = 0x10:0xffffffffb1eac7b0
> frame pointer           = 0x10:0xffffffffb1eac7e0
> code segment            = base 0x0, limit 0xfffff, type 0x1b
>                         = DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags        = interrupt enabled, resume, IOPL = 0
> current process         = 213 (sh)
> [thread pid 213 tid 100066 ]
> Stopped at      _mtx_lock_flags+0x35:   cmpq    $0x80779d40,0(%rdi)

unintialized mutex, probably, although it looks like it'd be the vm page
queue mutex which should be init'd by then.

Is this -CURRENT?

> db> tr
> Tracing pid 213 tid 100066 td 0xffffff007b9b1000
> _mtx_lock_flags() at _mtx_lock_flags+0x35
> exec_map_first_page() at exec_map_first_page+0x60

If you have a debug kernel for this around, load it into gdb and 'disass
exec_map_first_page' and look around offset 96 to see if its referencing a
mutex (mtx) near there.

> kern_execve() at kern_execve+0x2a0
> execve() at execve+0x5d
> syscall() at syscall+0x4ab
> Xfast_syscall() at Xfast_syscall+0xa8
> --- syscall (59, FreeBSD ELF64, execve), rip = 0x80090630c, rsp =
> 0x7fffffffcbf8, rbp = 0 ---
>
> doing the unionfs stuff sometime later - after the single user prompt - seems
> do be ok.
>
> another thing, there are some LOR caused by the unionfs & md:
> extarct of rc.d/initdiskless:
>
> if [ -e /conf/union ]; then
>     if ! sysctl vfs.uniondebug >/dev/null 2>&1; then
> 	echo Loading unionfs
> 	kldload unionfs
>     fi
>     echo Doing UNIONFS
>     mount_md 4096 /conf/etc
>     chmod 755 /conf/etc
>     mount_unionfs /conf/etc /etc
>     ls -R /etc > /dev/null
>     touch /etc/.sentinel
>     md_created_etc=created
> fi
> ...
>
> Loading unionfs
> lock order reversal
>  1st 0xffffff007c3b00f0 system map (system map) _at_ /r+d/6.0/src/sys/vm/vm_map.c:
> 1100
>  2nd 0xffffff00623d8620 vm object (standard object) _at_
> /r+d/6.0/src/sys/vm/vm_map.c:897
> KDB: stack backtrace:
> witness_checkorder() at witness_checkorder+0x5f1
> _mtx_lock_flags() at _mtx_lock_flags+0x4a
> vm_map_insert() at vm_map_insert+0x115
> vm_map_find() at vm_map_find+0x9d
> link_elf_load_file() at link_elf_load_file+0x811
> linker_load_module() at linker_load_module+0x50e
> kldload() at kldload+0xc3
> syscall() at syscall+0x4ab
> Xfast_syscall() at Xfast_syscall+0xa8
> --- syscall (304, FreeBSD ELF64, kldload), rip = 0x80067950c, rsp =
> 0x7fffffffedf8, rbp = 0x7fffffffee68 ---
>
> Doing UNIONFS
> lock order reversal
>  1st 0xffffffff80892040 vm page queue mutex (vm page queue mutex) _at_
> /r+d/6.0/src/sys/kern/vfs_bio.c:1485
>  2nd 0xffffff0061dc98b0 vnode interlock (vnode interlock) _at_
> /r+d/6.0/src/sys/kern/vfs_subr.c:1992
> KDB: stack backtrace:
> witness_checkorder() at witness_checkorder+0x5f1
> _mtx_lock_flags() at _mtx_lock_flags+0x4a
> vdrop() at vdrop+0x31
> vm_page_remove() at vm_page_remove+0x126
> vm_page_free_toq() at vm_page_free_toq+0x61
> vm_page_free() at vm_page_free+0x1e
> vfs_vmio_release() at vfs_vmio_release+0x18f
> brelse() at brelse+0x792
> ffs_mount() at ffs_mount+0x121b
> vfs_donmount() at vfs_donmount+0xaa3
> kernel_mount() at kernel_mount+0xaf
> ffs_cmount() at ffs_cmount+0x92
> mount() at mount+0x132
> syscall() at syscall+0x1fb
> Xfast_syscall() at Xfast_syscall+0xa8
> --- syscall (21, FreeBSD ELF64, mount), rip = 0x80067e68c, rsp =
> 0x7fffffffdf98, rbp = 0x7fffffffea58 ---
>
>
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>

-- 
Doug White                    |  FreeBSD: The Power to Serve
dwhite_at_gumbysoft.com          |  www.FreeBSD.org