Re: watchdogd coredump

On Sun, 4 Nov 2012 00:08:43 +0200 Konstantin Belousov
<kostikbel_at_gmail.com> wrote:

> On Sat, Nov 03, 2012 at 10:55:42PM +0100, Alexander Leidinger wrote:
> > Hi,
> > 
> > I updated from r239708 to r242511 and my watchdogd coredumps (and
> > brings down the system... well, the WD works).
> > 
> > Before I have a deeper look (recompiling with debugging and such) at
> > this, can someone confirm that with a recent -current and a WD
> > configured to call a shell script which does a simple ls
> > to /dev/null there is a segfault in watchdogd?
> > 
> > watchdogd_flags="-e /root/bin/wd_check.sh -s 5 -t 60"
> > 
> > ---snip---
> > # cat /root/bin/wd_check.sh
> > #!/bin/sh
> > 
> > exec ls / /space/jails >/dev/null 2>&1 </dev/null
> > ---snip---
> > 
> > The not so useful backtrace of the watchdogd core:
> > ---snip---
> > #0  0x0000000800609520 in dlopen () from /libexec/ld-elf.so.1
> > #1  0x000000080060309d in .text () from /libexec/ld-elf.so.1
> > #2  0x0000000000000246 in ?? ()
> > #3  0xfffffe001ee700f8 in ?? ()
> > #4  0xfffffe001ee700f8 in ?? ()
> > #5  0xffffffff80647e80 in ?? ()
> > #6  0x0000000000000001 in ?? ()
> > #7  0x00007fffffffdb90 in ?? ()
> > #8  0x0000000000000013 in ?? ()
> > #9  0x00007fffffffdb80 in ?? ()
> > #10 0x0000000000000000 in ?? ()
> > #11 0x0000000000000206 in ?? ()
> > #12 0x00007fffffffdaf0 in ?? ()
> > #13 0x000000080061cc00 in ?? ()
> > #14 0x0000000000000031 in ?? ()
> > #15 0x0000000800c93dc9 in system () from /lib/libc.so.7
> > #16 0x0000000000401558 in ?? ()
> > #17 0x0000000000400f7e in ?? ()
> > #18 0x000000080061c000 in ?? ()
> > #19 0x0000000000000000 in ?? ()
> > #20 0x0000000000000007 in ?? ()
> > #21 0x00007fffffffdef8 in ?? ()
> > ---snip---
> 
> Are you sure that your kernel is at r242511 ?
> 
> The issue should have been fixed by r242011.

I got some time again to look into this. I recompiled the libs with
debugging symbols and installed unstripped versions, but I don't get an
useful backtrace:
---snip---
# gdb /usr/sbin/watchdogd /watchdogd.core
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and
you are welcome to change it and/or distribute copies of it under
certain conditions. Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for
details. This GDB was configured as "amd64-marcel-freebsd"...(no
debugging symbols found)... Core was generated by `watchdogd'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libm.so.5...done.
Loaded symbols for /lib/libm.so.5
Reading symbols from /lib/libutil.so.9...done.
Loaded symbols for /lib/libutil.so.9
Reading symbols from /lib/libc.so.7...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /libexec/ld-elf.so.1...done.
Loaded symbols for /libexec/ld-elf.so.1
#0  0x0000000800609520 in dlopen () from /libexec/ld-elf.so.1
(gdb) bt
#0  0x0000000800609520 in dlopen () from /libexec/ld-elf.so.1
#1  0x000000080060309d in .text () from /libexec/ld-elf.so.1
#2  0x0000000000000246 in ?? ()
#3  0xfffffe0179f060f8 in ?? ()
#4  0xfffffe0179f060f8 in ?? ()
#5  0xffffffff80647e80 in ?? ()
#6  0x0000000000000001 in ?? ()
#7  0x00007fffffffdb90 in ?? ()
#8  0x0000000000000013 in ?? ()
#9  0x00007fffffffdb80 in ?? ()
#10 0x0000000000000000 in ?? ()
#11 0x0000000000000206 in ?? ()
#12 0x00007fffffffdaf0 in ?? ()
#13 0x000000080061cc00 in ?? ()
#14 0x0000000000000031 in ?? ()
#15 0x0000000800c93dc9 in __system (
    command=0x801406040 "/root/bin/wd_check.sh")
    at /space/system/usr_src/lib/libc/stdlib/system.c:70
#16 0x0000000000401558 in main ()


# uname  -v
FreeBSD 10.0-CURRENT #10 r242511M: Sat Nov  3 17:49:09 CET 2012 
---snip---

How can I debug this further?

Bye,
Alexander.

-- 
http://www.Leidinger.net    Alexander _at_ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org       netchild _at_ FreeBSD.org  : PGP ID = 72077137