Re: malloc() call somehow calling the rtld malloc() implementaion

On Fri, 27 Jan 2017 00:31:30 -0800 (PST)
Don Lewis <truckman_at_FreeBSD.org> wrote:

> I've been attempting to get OpenOffice to build properly in a
> clang400-import poudriere jail and have run into a mystery.  The build
> procedure creates a c++ executable "idlc", which is used to build
> other stuff.  The new operator has been overrriden to call a custom
> memory allocator, which I have configured to call the system version
> of malloc().
> 
> At some point idlc crashes because it has allocated a 16 byte
> structure and the compiler is using "movaps %xmm0,(%rax)" to
> initialize it, which requires 16 byte alignment.  Unfortunately this
> structure is only 8 byte aligned, causing a bus error.  This
> shouldn't be happening because our system malloc() always seems to do
> the proper alignment.  It appears that intead of calling the version
> of malloc() in libc, the simple version of malloc() built into rtld
> is being called instead.
> 
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and
> you are welcome to change it and/or distribute copies of it under
> certain conditions. Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for
> details. This GDB was configured as "amd64-marcel-freebsd"...
> (gdb) break main
> Breakpoint 1 at 0x43b1f6: file idlcmain.cxx, line 34.
> (gdb) run _at_/tmp/r
> Starting
> program: /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/bin/idlc
> _at_/tmp/r [New LWP 101304] [New Thread 802616000 (LWP 101304/idlc)]
> [Switching to Thread 802616000 (LWP 101304/idlc)]
> 
> Breakpoint 1, main (argc=2, argv=0x7fffffffb278) at idlcmain.cxx:34
> 34	SAL_IMPLEMENT_MAIN_WITH_ARGS(argc, argv)
> (gdb) break malloc
> Breakpoint 2 at 0x8006a5f01:
> file /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c,
> line 163. (gdb) cont Continuing.
> Trace 12474/1: "Min Prioriy for policy '2' == '0'
> "
> Trace 12474/1: "Max Prioriy for policy '2' == '103'
> "
> /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/bin/idlc:
> compiling 1 source files ...
> Compiling: /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/offapi/com/sun/star/i18n/KParseTokens.idl
> 
> Breakpoint 2, malloc (nbytes=343)
>     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
> 163	/var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:
> No such file or directory.
> in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
> Current language:  auto; currently minimal (gdb) cont
> Continuing.
> 
> Breakpoint 2, malloc (nbytes=32)
>     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
> 163
> in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
> (gdb) cont Continuing.
> Trace 13112/2: "ChildStatusProc : starting
> '/wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/bin/ucpp'" [New
> Thread 802616500 (LWP 100249/idlc)] [Switching to Thread 802616500
> (LWP 100249/idlc)]
> 
> Breakpoint 2, malloc (nbytes=19)
>     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
> 163
> in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
> (gdb) cont Continuing.
> 
> Breakpoint 2, malloc (nbytes=34)
>     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
> 163
> in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
> (gdb) cont Continuing.
> 
> Breakpoint 2, malloc (nbytes=16)
>     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
> 163
> in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
> (gdb) cont Continuing.
> 
> Breakpoint 2, malloc (nbytes=16)
>     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
> 163
> in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
> (gdb) cont Continuing.
> 
> Breakpoint 2, malloc (nbytes=16)
>     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
> 163
> in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
> (gdb) cont Continuing.
> 
> Breakpoint 2, malloc (nbytes=16)
>     at /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c:163
> 163
> in /var/poudriere/jails/clang400amd64/usr/src/libexec/rtld-elf/malloc.c
> (gdb) cont Continuing.
> sizeof(AstExprValue)=16
> 
> Program received signal SIGBUS, Bus error.
> [Switching to Thread 802616000 (LWP 101304/idlc)]
> 0x0000000000478cc2 in AstExpression::eval_bit_op (this=0x802633dc8, 
>     ek=EK_const) at astexpression.cxx:1001
> 1001	    std::auto_ptr< AstExprValue	> retval(new
> AstExprValue());
> 
> 
> idlc is linked to libc.so:
> 
> /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/bin/idlc:
> 	libreg.so.3
> => /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/lib/libreg.so.3
> (0x8008b9000) libuno_sal.so.3
> => /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/lib/libuno_sal.so.3
> (0x800c00000) libuno_salhelpergcc3.so.3
> => /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/lib/libuno_salhelpergcc3.so.3
> (0x801040000) libm.so.5 => /lib/libm.so.5 (0x801244000) libc++.so.1
> => /usr/lib/libc++.so.1 (0x80146e000) libcxxrt.so.1
> => /lib/libcxxrt.so.1 (0x801735000) libgcc_s.so.1
> => /lib/libgcc_s.so.1 (0x801953000) libthr.so.3 => /lib/libthr.so.3
> (0x801b69000) libc.so.7 => /lib/libc.so.7 (0x801d91000) libstore.so.3
> => /wrkdirs/usr/ports/editors/openoffice-devel/work/aoo-4.2.0/main/solver/420/unxfbsdx.pro/lib/libstore.so.3
> (0x802152000)
> 
> 
> If I create a simple test program that calls malloc() and set a
> breakpoint in malloc(), the breakpoint gets set in the rtld version,
> but the the libc version of malloc is what gets called.
> 
> What the heck is going on here, and how can I fix it?
> 

rtld on my system does not have malloc exposed as dynamic symbol, it
cannot possibly be used for symbol resolution by any outside module.

-- 
Alexander Kabaev