Re: C++ in jemalloc

From: Mark Millard <markmi_at_dsl-only.net>
Date: Sat, 7 Oct 2017 03:13:43 -0700
Just a short top-post as this does not fit well with the
other material:

I believe Roman only built his example program
with clang, not the world that the program was
being run under.

The gcc 4.2.1 based code that is analogous to
__cxa_begin_catch (scratch register initialization)
in a clang based build world has the scratch
register CFI material and the same clang produced
a.out file works fine under such a system (simply
copied over and run).

And that is why Roman's context did not SIGSEGV but
mine did: I used clang for the buildworld for
the world that was being used and so
__cxa_begin_catch was missing CFI information in
my build.

In fact the a.out built by gcc 4.2.1 fails under
the clang based buildworld with the bad
__cxa_begin_catch .

The only thing that matters is the system library
code involved, not which a.out (from which compiler).


On 2017-Oct-7, at 2:54 AM, Mark Millard <markmi at dsl-only.net> wrote:

[The last part of my note has a dumb mistake, not
that it messes up the other evidence. I should have
dwarfdump'd not a.out but the code in the libraries,
such as __cxa_begin_catch in /lib/libcxxrt.so.1 . I
made the same mistake initially back when Roman and
I were dealing with this long ago. Correcting it
again. . .]

On 2017-Oct-7, at 2:18 AM, Mark Millard <markmi at dsl-only.net> wrote:

> [I'm separately listing backtrace information and related
> information from one of core dumps and going back through
> the details to see if they seem to be as they were back
> then. Read only if you care. It does look the same as I
> found back then if I remember right. I reach the same
> conclusion I reached back then anyway. I give evidence
> in the below.]
> 
> On 2017-Oct-7, at 1:10 AM, Mark Millard <markmi at dsl-only.net> wrote:
> 
>> [I'm adding examples with output from clang -v since it explicitly shows
>> the path used for ld and such.]
>> 
>> On 2017-Oct-7, at 12:58 AM, Mark Millard <markmi at dsl-only.net> wrote:
>> 
>>> On 2017-Oct-6, at 11:42 PM, Roman Divacky <rdivacky at vlakno.cz> wrote:
>>> 
>>>> Just to clarify my not agreeing with Mark regarding EH on ppc64.
>>>> 
>>>> Last time I tried to fix ppc64 exceptions handling as generated by clang
>>>> it turned out that simply using gnu ld from ports fixes the issue.
>>>> 
>>>> For details see:
>>>> https://lists.freebsd.org/pipermail/freebsd-toolchain/2017-May/002961.html
>>> 
>>> Unfortunately my experiments failed to confirm this. Repeating
>>> them now under head -r324071 and ports -r450478 :
>>> 
>>> # more exception_test.cpp 
>>> #include <exception>
>>> 
>>> int main(void)
>>> {
>>> try { throw std::exception(); }
>>> catch (std::exception& e) {}
>>> return 0;
>>> }
>>> 
>>> # clang++ -B /usr/local/powerpc64-freebsd/bin -std=c++14 -g -O2 exception_test.cpp
>>> 
>>> # ./a.out
>>> Segmentation fault (core dumped)
>>> 
>>> # clang++ -B /usr/local/powerpc64-freebsd/bin -std=c++11 -g exception_test.cpp
>>> 
>>> # ./a.out
>>> Segmentation fault (core dumped)
>> 
>> # clang++ -v -B /usr/local/powerpc64-freebsd/bin -std=c++11 -g exception_test.cpp
>> FreeBSD clang version 5.0.0 (tags/RELEASE_500/final 312559) (based on LLVM 5.0.0svn)
>> Target: powerpc64-unknown-freebsd12.0
>> Thread model: posix
>> InstalledDir: /usr/bin
>> "/usr/bin/clang++" -cc1 -triple powerpc64-unknown-freebsd12.0 -emit-obj -mrelax-all -disable-free -main-file-name exception_test.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -masm-verbose -mconstructor-aliases -target-cpu ppc64 -mfloat-abi hard -v -dwarf-column-info -debug-info-kind=standalone -dwarf-version=2 -debugger-tuning=gdb -resource-dir /usr/lib/clang/5.0.0 -internal-isystem /usr/include/c++/v1 -std=c++11 -fdeprecated-macro -fdebug-compilation-dir /root/c_tests -ferror-limit 19 -fmessage-length 200 -fno-signed-char -fobjc-runtime=gnustep -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/exception_test-ba79a4.o -x c++ exception_test.cpp
>> clang -cc1 version 5.0.0 based upon LLVM 5.0.0svn default target powerpc64-unknown-freebsd12.0
>> #include "..." search starts here:
>> #include <...> search starts here:
>> /usr/include/c++/v1
>> /usr/lib/clang/5.0.0/include
>> /usr/include
>> End of search list.
>> "/usr/local/powerpc64-freebsd/bin/ld" --eh-frame-hdr -dynamic-linker /libexec/ld-elf.so.1 --enable-new-dtags -o a.out /usr/lib/crt1.o /usr/lib/crti.o /usr/lib/crtbegin.o -L/usr/lib /tmp/exception_test-ba79a4.o -lc++ -lm -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/crtend.o /usr/lib/crtn.o
>> 
>> # ./a.out
>> Segmentation fault (core dumped)
>> 
>> 
>>> # ls -lt /usr/local/powerpc64-freebsd/bin
>>> total 56704
>>> lrwxr-xr-x  1 root  wheel       32 Jul  2 19:27 size -> ../../bin/powerpc64-freebsd-size
>>> -r-xr-xr-x  4 root  wheel  7072791 Jul  2 19:27 ld
>>> -r-xr-xr-x  4 root  wheel  7072791 Jul  2 19:27 ld.bfd
>>> -r-xr-xr-x  2 root  wheel  6881822 Jul  2 19:27 as
>>> -r-xr-xr-x  2 root  wheel  6128889 Jul  2 19:27 strip
>>> -r-xr-xr-x  2 root  wheel  5253417 Jul  2 19:27 nm
>>> -r-xr-xr-x  2 root  wheel  1284139 Jul  2 19:27 readelf
>>> -r-xr-xr-x  2 root  wheel  6128882 Jul  2 19:27 objcopy
>>> -r-xr-xr-x  2 root  wheel  5384166 Jul  2 19:27 ranlib
>>> -r-xr-xr-x  2 root  wheel  5384159 Jul  2 19:27 ar
>>> -r-xr-xr-x  2 root  wheel  6914775 Jul  2 19:27 objdump
>>> 
>>> # clang++ -B /usr/local/powerpc64-portbld-freebsd12.0/bin/ -std=c++14 -g -O2 exception_test.cpp
>>> 
>>> # ./a.out
>>> Segmentation fault (core dumped)
>> 
>> # clang++ -v -B /usr/local/powerpc64-portbld-freebsd12.0/bin/ -std=c++14 -g -O2 exception_test.cpp
>> FreeBSD clang version 5.0.0 (tags/RELEASE_500/final 312559) (based on LLVM 5.0.0svn)
>> Target: powerpc64-unknown-freebsd12.0
>> Thread model: posix
>> InstalledDir: /usr/bin
>> "/usr/bin/clang++" -cc1 -triple powerpc64-unknown-freebsd12.0 -emit-obj -disable-free -main-file-name exception_test.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -mdisable-fp-elim -masm-verbose -mconstructor-aliases -target-cpu ppc64 -mfloat-abi hard -v -dwarf-column-info -debug-info-kind=standalone -dwarf-version=2 -debugger-tuning=gdb -resource-dir /usr/lib/clang/5.0.0 -internal-isystem /usr/include/c++/v1 -O2 -std=c++14 -fdeprecated-macro -fdebug-compilation-dir /root/c_tests -ferror-limit 19 -fmessage-length 200 -fno-signed-char -fobjc-runtime=gnustep -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops -vectorize-slp -o /tmp/exception_test-3ebf72.o -x c++ exception_test.cpp
>> clang -cc1 version 5.0.0 based upon LLVM 5.0.0svn default target powerpc64-unknown-freebsd12.0
>> #include "..." search starts here:
>> #include <...> search starts here:
>> /usr/include/c++/v1
>> /usr/lib/clang/5.0.0/include
>> /usr/include
>> End of search list.
>> "/usr/local/powerpc64-portbld-freebsd12.0/bin/ld" --eh-frame-hdr -dynamic-linker /libexec/ld-elf.so.1 --enable-new-dtags -o a.out /usr/lib/crt1.o /usr/lib/crti.o /usr/lib/crtbegin.o -L/usr/lib /tmp/exception_test-3ebf72.o -lc++ -lm -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/crtend.o /usr/lib/crtn.o
>> 
>> # ./a.out
>> Segmentation fault (core dumped)
>> 
>>> # clang++ -B /usr/local/powerpc64-portbld-freebsd12.0/bin/ -std=c++11 -g exception_test.cpp
>>> 
>>> # ./a.out
>>> Segmentation fault (core dumped)
>>> 
>>> 
>>> # ls -lt /usr/local/powerpc64-portbld-freebsd12.0/bin/
>>> total 363584
>>> -r-xr-xr-x  4 root  wheel  59993201 Jul  2 23:44 ld
>>> -r-xr-xr-x  4 root  wheel  59993201 Jul  2 23:44 ld.bfd
>>> -r-xr-xr-x  2 root  wheel  29843304 Jul  2 23:44 as
>>> -r-xr-xr-x  2 root  wheel  29046519 Jul  2 23:44 strip
>>> -r-xr-xr-x  2 root  wheel  28207257 Jul  2 23:44 nm
>>> -r-xr-xr-x  2 root  wheel   1178483 Jul  2 23:44 readelf
>>> -r-xr-xr-x  1 root  wheel  28329180 Jul  2 23:44 dlltool
>>> -r-xr-xr-x  2 root  wheel  29046512 Jul  2 23:44 objcopy
>>> -r-xr-xr-x  2 root  wheel  28334599 Jul  2 23:44 ranlib
>>> -r-xr-xr-x  2 root  wheel  28334592 Jul  2 23:44 ar
>>> -r-xr-xr-x  2 root  wheel  49540244 Jul  2 23:44 objdump
> 
> [Note: I used /usr/libexec/gdb because /usr/local/bin/gdb
> core dumps for this. powerpc64 is an example of the
> devel/gdb port currently being worse than the system gdb
> in various cases.]
> 
> # /usr/libexec/gdb a.out /var/crash/a.out.12775.core 
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "powerpc64-marcel-freebsd"...
> Core was generated by `./a.out'.
> Program terminated with signal 11, Segmentation fault.
> Reading symbols from /usr/lib/libc++.so.1...Reading symbols from /usr/lib/debug//usr/lib/libc++.so.1.debug...done.
> done.
> . . .
> Loaded symbols for /libexec/ld-elf.so.1
> #0  0x0000000050530c20 in _Unwind_SetGR (context=<value optimized out>, index=<value optimized out>, val=1342447712) at unwind-dw2-fde.h:162
> 162	{
> (gdb) bt
> #0  0x0000000050530c20 in _Unwind_SetGR (context=<value optimized out>, index=<value optimized out>, val=1342447712) at unwind-dw2-fde.h:162
> #1  0x0000000050190194 in __gxx_personality_v0 (version=<value optimized out>, actions=<value optimized out>, exceptionClass=<value optimized out>, exceptionObject=0x50042060, 
>   context=0xffffffffffffcc70) at /usr/src/contrib/libcxxrt/exception.cc:1203
> #2  0x0000000050531a60 in _Unwind_RaiseException_Phase2 (exc=0x50042060, context=0xffffffffffffcc70) at unwind.inc:66
> #3  0x0000000050531548 in _Unwind_RaiseException (exc=<value optimized out>) at unwind.inc:135
> #4  0x000000005018f4f4 in __cxa_throw (thrown_exception=<value optimized out>, tinfo=<value optimized out>, dest=<value optimized out>) at /usr/src/contrib/libcxxrt/exception.cc:774
> #5  0x0000000010000d74 in main () at exception_test.cpp:5
> #6  0x0000000010000ae8 in _start (argc=1342447624, argv=0x50042060, env=0xffffffffffffcc70, obj=<value optimized out>, cleanup=<value optimized out>, ps_strings=<value optimized out>)
>   at /usr/src/lib/csu/powerpc64/crt1.c:94
> #7  0x0000000050017b70 in .text () at /usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104
> 
> . . .
> 
> __gxx_personality_v0 was trying to execute:
> 
>       _Unwind_SetGR(context, __builtin_eh_return_data_regno(0),
>                     reinterpret_cast<unsigned long>(exceptionObject));
> 
> That was part of the sequence:
> 
> . . .
>       _Unwind_SetIP(context, reinterpret_cast<unsigned long>(action.landing_pad));
>       _Unwind_SetGR(context, __builtin_eh_return_data_regno(0),
>                     reinterpret_cast<unsigned long>(exceptionObject));
>       _Unwind_SetGR(context, __builtin_eh_return_data_regno(1), selector);
> 
>       return _URC_INSTALL_CONTEXT;
> 
> 
> So it died trying to use the CFA information for one of
> the scratch registers, the one used for the "exceptionObject".
> 
> The details are:
> 
> inline void
> _Unwind_SetGR (struct _Unwind_Context *context, int index, _Unwind_Word val)
> {
> int size;
> void *ptr;
> 
> index = DWARF_REG_TO_UNWIND_COLUMN (index);
> gcc_assert (index < (int) sizeof(dwarf_reg_size_table));
> size = dwarf_reg_size_table[index];
> 
> if (_Unwind_IsExtendedContext (context) && context->by_value[index])
>   {
>     context->reg[index] = (void *) (_Unwind_Internal_Ptr) val;
>     return;
>   }
> 
> ptr = context->reg[index];
> 
> if (size == sizeof(_Unwind_Ptr))
>   * (_Unwind_Ptr *) ptr = val;
> else
>   {
>     gcc_assert (size == sizeof(_Unwind_Word));
>     * (_Unwind_Word *) ptr = val;
>   }
> }
> 
> Note: DWARF_REG_TO_UNWIND_COLUMN leaves the index value
> unchanged for the value involved. I'll not provide the
> evidence here.
> 
> Breakpoint 1, main () at exception_test.cpp:5
> 5	    try { throw std::exception(); }
> (gdb) br _Unwind_SetGR
> Breakpoint 2 at 0x50530bbc
> (gdb) c
> Continuing.
> 
> Breakpoint 2, _Unwind_SetGR (context=0xffffffffffffcc30, index=3, val=1342447712) at /usr/src/gnu/lib/libgcc/../../../contrib/gcc/unwind-dw2.c:207
> 207	  index = DWARF_REG_TO_UNWIND_COLUMN (index);
> Current language:  auto; currently minimal
> (gdb) bt
> #0  _Unwind_SetGR (context=0xffffffffffffcc30, index=3, val=1342447712) at /usr/src/gnu/lib/libgcc/../../../contrib/gcc/unwind-dw2.c:207
> #1  0x0000000050190194 in __gxx_personality_v0 (version=<value optimized out>, actions=<value optimized out>, exceptionClass=<value optimized out>, exceptionObject=0x50042060, 
>   context=0xffffffffffffcc30) at /usr/src/contrib/libcxxrt/exception.cc:1203
> #2  0x0000000050531a60 in _Unwind_RaiseException_Phase2 (exc=0x50042060, context=0xffffffffffffcc30) at unwind.inc:66
> #3  0x0000000050531548 in _Unwind_RaiseException (exc=<value optimized out>) at unwind.inc:135
> #4  0x000000005018f4f4 in __cxa_throw (thrown_exception=<value optimized out>, tinfo=<value optimized out>, dest=<value optimized out>) at /usr/src/contrib/libcxxrt/exception.cc:774
> #5  0x0000000010000d74 in main () at exception_test.cpp:5
> #6  0x0000000010000ae8 in _start (argc=1342447624, argv=0x50042060, env=0xffffffffffffcc30, obj=<value optimized out>, cleanup=<value optimized out>, ps_strings=<value optimized out>)
>   at /usr/src/lib/csu/powerpc64/crt1.c:94
> #7  0x0000000050017b70 in .text () at /usr/src/libexec/rtld-elf/powerpc64/rtld_start.S:104
> (gdb) print *context
> $2 = {reg = 0xffffffffffffcc30, cfa = 0xffffffffffffd990, ra = 0x10000d78, lsda = 0x10000f74, bases = {tbase = 0x0, dbase = 0x0, func = 0x10000d30}, flags = 4611686018427387904, version = 0, 
> args_size = 0, by_value = 0xffffffffffffd108 ""}
> (gdb) print dwarf_reg_size_table[3]
> $3 = 8 '\b'
> (gdb) print context->reg[3]
> $4 = (void *) 0x0
> 
> And there is the problem: lack of the value
> that is supposed to be available there: NULL
> instead.
> 
> This traces back to. . .

. . . (removed mistake) . . .

Quoting an old note to Roman:

The landing pad area includes the __cxa_being_catch and
__cxa_end_catch and the involved code from
/lib/libcxxrt.so.1 should have the scratch-register CFI
entries, not the above main code.

So there was no point to my listing the CFI information
for main.

End Quote.

The below points out where there should have been
CFI information for the scratch registers, including
the failing r3 use. (Below is essentially my self-
correction text that I sent to Roman long ago.)

# ldd a.out
a.out:
	libc++.so.1 => /usr/lib/libc++.so.1 (0x5004e000)
	libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x5017b000)
	libm.so.5 => /lib/libm.so.5 (0x501a2000)
	libc.so.7 => /lib/libc.so.7 (0x501da000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x5052c000)

The used scratch registers (of r3, r4, r5, r6) should
show up in:

# dwarfdump -v -v -F /lib/libcxxrt.so.1  | egrep -i "\<r[3456]\>"
#

but do not.

In __cxa_begin_catch below note the code setting up r3-r5
before the branch to __cxa_begin_catch+164 and that
__cxa_begin_catch+164 on is exit code that leaves
these scratch register alone. (The path skipping
__cxa_begin_catch+144 is similar for the issue.)

__cxa_being_catch's CFI information should list the
r3, r4, and r5 scratch register usage but does not.
(Powerpc 32-bit has the same issue for the same
registers for its __cxa_begin_catch .)


(gdb) Dump of assembler code for function __cxa_begin_catch:
0x000000005018ecc4 <__cxa_begin_catch+0>:	mflr    r0
0x000000005018ecc8 <__cxa_begin_catch+4>:	std     r0,16(r1)
0x000000005018eccc <__cxa_begin_catch+8>:	stdu    r1,-144(r1)
0x000000005018ecd0 <__cxa_begin_catch+12>:	std     r29,120(r1)
0x000000005018ecd4 <__cxa_begin_catch+16>:	std     r30,128(r1)
0x000000005018ecd8 <__cxa_begin_catch+20>:	mr      r29,r3
0x000000005018ecdc <__cxa_begin_catch+24>:	bl      0x5018e634 <_ZL11thread_infov>
0x000000005018ece0 <__cxa_begin_catch+28>:	mr      r30,r3
0x000000005018ece4 <__cxa_begin_catch+32>:	lwz     r3,48(r30)
0x000000005018ece8 <__cxa_begin_catch+36>:	addi    r3,r3,-1
0x000000005018ecec <__cxa_begin_catch+40>:	stw     r3,48(r30)
0x000000005018ecf0 <__cxa_begin_catch+44>:	ld      r3,0(r29)
0x000000005018ecf4 <__cxa_begin_catch+48>:	bl      0x50190210 <_ZL14isCXXExceptionm>
0x000000005018ecf8 <__cxa_begin_catch+52>:	cmpldi  r3,0
0x000000005018ecfc <__cxa_begin_catch+56>:	beq-    0x5018ed44 <__cxa_begin_catch+128>
0x000000005018ed00 <__cxa_begin_catch+60>:	mr      r3,r29
0x000000005018ed04 <__cxa_begin_catch+64>:	bl      0x5018f81c <_ZL20exceptionFromPointerPv>
0x000000005018ed08 <__cxa_begin_catch+68>:	lwz     r4,48(r3)
0x000000005018ed0c <__cxa_begin_catch+72>:	cmplwi  r4,0
0x000000005018ed10 <__cxa_begin_catch+76>:	bne-    0x5018ed20 <__cxa_begin_catch+92>
0x000000005018ed14 <__cxa_begin_catch+80>:	ld      r5,40(r30)
0x000000005018ed18 <__cxa_begin_catch+84>:	std     r5,40(r3)
0x000000005018ed1c <__cxa_begin_catch+88>:	std     r3,40(r30)
0x000000005018ed20 <__cxa_begin_catch+92>:	srawi   r5,r4,31
0x000000005018ed24 <__cxa_begin_catch+96>:	li      r29,0
0x000000005018ed28 <__cxa_begin_catch+100>:	add     r4,r4,r5
0x000000005018ed2c <__cxa_begin_catch+104>:	xor     r4,r4,r5
0x000000005018ed30 <__cxa_begin_catch+108>:	addi    r4,r4,1
0x000000005018ed34 <__cxa_begin_catch+112>:	stw     r4,48(r3)
0x000000005018ed38 <__cxa_begin_catch+116>:	stw     r29,32(r30)
0x000000005018ed3c <__cxa_begin_catch+120>:	ld      r3,80(r3)
0x000000005018ed40 <__cxa_begin_catch+124>:	b       0x5018ed68 <__cxa_begin_catch+164>
0x000000005018ed44 <__cxa_begin_catch+128>:	ld      r3,40(r30)
0x000000005018ed48 <__cxa_begin_catch+132>:	cmpldi  r3,0
0x000000005018ed4c <__cxa_begin_catch+136>:	beq-    0x5018ed58 <__cxa_begin_catch+148>
0x000000005018ed50 <__cxa_begin_catch+140>:	bl      0x50184480 <00000017.plt_call._ZSt9terminatev>
0x000000005018ed54 <__cxa_begin_catch+144>:	ld      r2,40(r1)
0x000000005018ed58 <__cxa_begin_catch+148>:	addi    r3,r29,32
0x000000005018ed5c <__cxa_begin_catch+152>:	li      r4,1
0x000000005018ed60 <__cxa_begin_catch+156>:	std     r29,40(r30)
0x000000005018ed64 <__cxa_begin_catch+160>:	stw     r4,32(r30)
0x000000005018ed68 <__cxa_begin_catch+164>:	ld      r30,128(r1)
0x000000005018ed6c <__cxa_begin_catch+168>:	ld      r29,120(r1)
0x000000005018ed70 <__cxa_begin_catch+172>:	addi    r1,r1,144
0x000000005018ed74 <__cxa_begin_catch+176>:	ld      r0,16(r1)
0x000000005018ed78 <__cxa_begin_catch+180>:	mtlr    r0
0x000000005018ed7c <__cxa_begin_catch+184>:	blr
0x000000005018ed80 <__cxa_begin_catch+188>:	.long 0x0
0x000000005018ed84 <__cxa_begin_catch+192>:	.long 0x0
0x000000005018ed88 <__cxa_begin_catch+196>:	.long 0x0
End of assembler dump.


Sorry for the earlier mis-direction.


===
Mark Millard
markmi at dsl-only.net
Received on Sat Oct 07 2017 - 08:13:46 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:13 UTC