Re: Crashes in libthr?

From: Larry Rosenman <ler_at_lerctr.org>
Date: Sun, 13 Mar 2016 14:10:58 -0500
On 2016-03-13 13:58, Konstantin Belousov wrote:
> On Sun, Mar 13, 2016 at 01:32:20PM -0500, Larry Rosenman wrote:
>> On 2016-03-13 13:12, Konstantin Belousov wrote:
>> > On Sun, Mar 13, 2016 at 11:16:20AM -0500, Larry Rosenman wrote:
>> >> I updated one of my servers, and WHILE DOING THE INSTALLWORLD, I get
>> >> segfaults.
>> >>
>> >> ANY multithreaded program crashes.
>> >>
>> >> I reverted libthr, and it's fine.
>> >>
>> >> borg.lerctr.org / # gdb -c zfs.core /sbin/zfs
>> >> GNU gdb 6.1.1 [FreeBSD]
>> >> Copyright 2004 Free Software Foundation, Inc.
>> >> GDB is free software, covered by the GNU General Public License, and
>> >> you
>> >> are
>> >> welcome to change it and/or distribute copies of it under certain
>> >> conditions.
>> >> Type "show copying" to see the conditions.
>> >> There is absolutely no warranty for GDB.  Type "show warranty" for
>> >> details.
>> >> This GDB was configured as "amd64-marcel-freebsd"...
>> >> Core was generated by `zfs'.
>> >> Program terminated with signal 11, Segmentation fault.
>> >> Reading symbols from /lib/libjail.so.1...Reading symbols from
>> >> /usr/lib/debug//lib/libjail.so.1.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libjail.so.1
>> >> Reading symbols from /lib/libnvpair.so.2...Reading symbols from
>> >> /usr/lib/debug//lib/libnvpair.so.2.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libnvpair.so.2
>> >> Reading symbols from /lib/libuutil.so.2...Reading symbols from
>> >> /usr/lib/debug//lib/libuutil.so.2.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libuutil.so.2
>> >> Reading symbols from /lib/libzfs_core.so.2...Reading symbols from
>> >> /usr/lib/debug//lib/libzfs_core.so.2.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libzfs_core.so.2
>> >> Reading symbols from /lib/libzfs.so.2...Reading symbols from
>> >> /usr/lib/debug//lib/libzfs.so.2.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libzfs.so.2
>> >> Reading symbols from /lib/libc.so.7...Reading symbols from
>> >> /usr/lib/debug//lib/libc.so.7.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libc.so.7
>> >> Reading symbols from /lib/libmd.so.6...Reading symbols from
>> >> /usr/lib/debug//lib/libmd.so.6.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libmd.so.6
>> >> Reading symbols from /lib/libumem.so.2...Reading symbols from
>> >> /usr/lib/debug//lib/libumem.so.2.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libumem.so.2
>> >> Reading symbols from /lib/libutil.so.9...Reading symbols from
>> >> /usr/lib/debug//lib/libutil.so.9.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libutil.so.9
>> >> Reading symbols from /lib/libm.so.5...Reading symbols from
>> >> /usr/lib/debug//lib/libm.so.5.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libm.so.5
>> >> Reading symbols from /lib/libavl.so.2...Reading symbols from
>> >> /usr/lib/debug//lib/libavl.so.2.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libavl.so.2
>> >> Reading symbols from /lib/libbsdxml.so.4...Reading symbols from
>> >> /usr/lib/debug//lib/libbsdxml.so.4.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libbsdxml.so.4
>> >> Reading symbols from /lib/libgeom.so.5...Reading symbols from
>> >> /usr/lib/debug//lib/libgeom.so.5.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libgeom.so.5
>> >> Reading symbols from /lib/libz.so.6...Reading symbols from
>> >> /usr/lib/debug//lib/libz.so.6.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libz.so.6
>> >> Reading symbols from /lib/libthr.so.3...done.
>> >> Loaded symbols for /lib/libthr.so.3
>> > Why all libs have debug symbols, while your most interesting one,
>> > libthr.so.3, does not ?
>> >
>> >> Reading symbols from /lib/libsbuf.so.6...Reading symbols from
>> >> /usr/lib/debug//lib/libsbuf.so.6.debug...done.
>> >> done.
>> >> Loaded symbols for /lib/libsbuf.so.6
>> >> Reading symbols from /libexec/ld-elf.so.1...done.
>> >> Loaded symbols for /libexec/ld-elf.so.1
>> >> #0  0x0000000802703f81 in __pthread_cxa_finalize () from
>> >> /lib/libthr.so.3
>> >> [New LWP 100957]
>> >> (gdb) bt
>> >> #0  0x0000000802703f81 in __pthread_cxa_finalize () from
>> >> /lib/libthr.so.3
>> >> #1  0x0000000802703e85 in __pthread_cxa_finalize () from
>> >> /lib/libthr.so.3
>> >> #2  0x0000000802707052 in ?? () from /lib/libthr.so.3
>> >> #3  0x000000080063fc00 in ?? ()
>> >> #4  0x00007fffffffe638 in ?? ()
>> >> #5  0x00007fffffffe5b0 in ?? ()
>> >> #6  0x00000008026f8fd6 in atoi_at_plt () from /lib/libthr.so.3
>> >> #7  0x00007fffffffe5b0 in ?? ()
>> >> #8  0x000000080061adfd in r_debug_state () from /libexec/ld-elf.so.1
>> >> Previous frame inner to this frame (corrupt stack?)
>> >> (gdb)
>> >>
>> >> old SVN: r296103
>> >> new SVN: r296796M
>> >> (The M is a nd6 patch from markj_at_)
>> >>
>> >> this was a FULL buildworld/buildkernel.
>> >
>> > If you cd lib/libthr and do
>> > 	make clean all install DEBUG_FLAGS=-g
>> > on the broken world, does it fix the problem ?  If not, do debugging
>> > symbols from libthr appear accessible to gdb at least ?  Try this to
>> > get useful backtrace with source lines information.
>> ar crashes linking the library......
> 
> ar does not link library, it archives .o into .a archive.
> That said, neither ar nor ld (I am not sure whether 'ar' or 'linking'
> is a thinko in the sentence above) do not depend on libthr. You have
> something more fundamental broken.
> 
> I put the libthr.so.3 built with the debugging symbols on amd64, using 
> head
> r296779, at https://people.freebsd.org/~kib/misc/libthr.so.3 .  Try 
> with
> it by manually copying the file to /lib.  If still crashing, it should
> show the line numbers.
no dice.

(gdb) borg.lerctr.org /home/ler $ sudo gdb -c ar.core /usr/bin/ar
Password:
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you 
are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for 
details.
This GDB was configured as "amd64-marcel-freebsd"...
Core was generated by `ar'.
Program terminated with signal 11, Segmentation fault.
#0  0x000000000042c271 in _thr_rtld_init ()
[New Thread 800a1b000 (LWP 100317/<unknown>)]
(gdb) bt
#0  0x000000000042c271 in _thr_rtld_init ()
#1  0x000000000042c19a in _libpthread_init ()
#2  0x00000000004eaa82 in __do_global_ctors_aux ()
#3  0x0000000000400196 in _init ()
#4  0x00007fffffffebd0 in ?? ()
#5  0x00000000004002a6 in _start ()
#6  0x0000000000000000 in ?? ()
(gdb)



-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640                 E-Mail: ler_at_lerctr.org
US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961
Received on Sun Mar 13 2016 - 18:10:58 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:03 UTC