Re: malloc() call somehow calling the rtld malloc() implementaion

From: Don Lewis <truckman_at_FreeBSD.org>
Date: Sat, 28 Jan 2017 01:56:16 -0800 (PST)
On 27 Jan, Alexander Kabaev wrote:
> On Fri, 27 Jan 2017 10:47:20 -0800 (PST)
> Don Lewis <truckman_at_FreeBSD.org> wrote:
> 
>> On 27 Jan, Alexander Kabaev wrote:
>> > On Fri, 27 Jan 2017 00:31:30 -0800 (PST)
>> > Don Lewis <truckman_at_FreeBSD.org> wrote:  
>> 
>> >> If I create a simple test program that calls malloc() and set a
>> >> breakpoint in malloc(), the breakpoint gets set in the rtld
>> >> version, but the the libc version of malloc is what gets called.
>> >> 
>> >> What the heck is going on here, and how can I fix it?
>> >>   
>> > 
>> > rtld on my system does not have malloc exposed as dynamic symbol, it
>> > cannot possibly be used for symbol resolution by any outside
>> > module.  
>> 
>> Same here, but gdb at least seems to find it anyway.
>> 
>> 12.0-CURRENT r311765M
>> 
>> %nm /libexec/ld-elf.so.1
>> nm: /libexec/ld-elf.so.1: no symbols
>> 
>> %elfdump -s /libexec/ld-elf.so.1 | grep st_name | sort
>> 	st_name: 
>> 	st_name: 
>> 	st_name: FBSD_1.0
>> 	st_name: FBSD_1.1
>> 	st_name: FBSD_1.2
>> 	st_name: FBSD_1.3
>> 	st_name: FBSD_1.4
>> 	st_name: FBSD_1.5
>> 	st_name: FBSDprivate_1.0
>> 	st_name: __tls_get_addr
>> 	st_name: _r_debug_postinit
>> 	st_name: _rtld_addr_phdr
>> 	st_name: _rtld_allocate_tls
>> 	st_name: _rtld_atfork_post
>> 	st_name: _rtld_atfork_pre
>> 	st_name: _rtld_error
>> 	st_name: _rtld_free_tls
>> 	st_name: _rtld_get_stack_prot
>> 	st_name: _rtld_is_dlopened
>> 	st_name: _rtld_thread_init
>> 	st_name: dl_iterate_phdr
>> 	st_name: dladdr
>> 	st_name: dlclose
>> 	st_name: dlerror
>> 	st_name: dlfunc
>> 	st_name: dlinfo
>> 	st_name: dllockinit
>> 	st_name: dlopen
>> 	st_name: dlsym
>> 	st_name: dlvsym
>> 	st_name: fdlopen
>> 	st_name: r_debug_state
>> 
>> %cd /tmp
>> zipper:/tmp 508%cat malloctest.c 
>> #include <stdlib.h>
>> volatile void *p;
>> int
>> main(void) {
>> 	p = malloc(16);
>> }
>> %cc -g malloctest.c
>> %gdb a.out
>> GNU gdb 6.1.1 [FreeBSD]
>> Copyright 2004 Free Software Foundation, Inc.
>> GDB is free software, covered by the GNU General Public License, and
>> you are welcome to change it and/or distribute copies of it under
>> certain conditions. Type "show copying" to see the conditions.
>> There is absolutely no warranty for GDB.  Type "show warranty" for
>> details. This GDB was configured as "amd64-marcel-freebsd"...
>> (gdb) break main
>> Breakpoint 1 at 0x40076b: file malloctest.c, line 5.
>> (gdb) run
>> Starting program: /tmp/a.out 
>> 
>> Breakpoint 1, main () at malloctest.c:5
>> 5		p = malloc(16);
>> Current language:  auto; currently minimal
>> (gdb) break malloc
>> Breakpoint 2 at 0x80060e9a4: file /usr/src/libexec/rtld-elf/malloc.c,
>> line 163. (gdb) cont
>> Continuing.
>> 
>> Program exited normally.
>> (gdb) quit
>> 
>> Ports gdb finds both the rtld malloc() and the libc malloc():
>> 
>> %/usr/local/bin/gdb a.out
>> GNU gdb (GDB) 7.12 [GDB v7.12 for FreeBSD]
>> Copyright (C) 2016 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later
>> <http://gnu.org/licenses/gpl.html> This is free software: you are
>> free to change and redistribute it. There is NO WARRANTY, to the
>> extent permitted by law.  Type "show copying" and "show warranty" for
>> details. This GDB was configured as "x86_64-portbld-freebsd12.0".
>> Type "show configuration" for configuration details.
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>.
>> Find the GDB manual and other documentation resources online at:
>> <http://www.gnu.org/software/gdb/documentation/>.
>> For help, type "help".
>> Type "apropos word" to search for commands related to "word"...
>> Reading symbols from a.out...done.
>> (gdb) break main
>> Breakpoint 1 at 0x40076b: file malloctest.c, line 5.
>> (gdb) run
>> Starting program: /tmp/a.out 
>> 
>> Breakpoint 1, main () at malloctest.c:5
>> 5		p = malloc(16);
>> (gdb) break malloc
>> Breakpoint 2 at 0x80060e9a4: malloc. (2 locations)
>> (gdb) cont
>> Continuing.
>> 
>> Breakpoint 2, __malloc (size=16) at jemalloc_jemalloc.c:1636
>> 1636		size_t usize JEMALLOC_CC_SILENCE_INIT(0);
> 
> gdb looks into debugging info for symbols in debug information and so
> has greater visibility. Symbols gdb sees play no role in dynasmic
> building, so rtld malloc somehow replacing the malloc from the program
> as you suspected is really impossible.

What threw me off here was that ld-elf.so is stripped, so I didn't think
that it had debug symbols available.  I'd forgotten that we now install
.debug files for the base system executables and shared libraries.

> kib_at_ has a better idea that is worth investigating further: is the
> address on which idlc traps corresponds to TLS storage? tls alloc calls
> are supposed to respect the tls section alignment though.

Once I had ports gdb available in the jail so that I could set
breakpoints in both versions of malloc(), I was able to see that all of
the rtld malloc() calls occurred before the start of main().  After
that, all of the malloc() calls used the libc version, and there were
many of those before the occurance of SIGBUS.

The problem turns out to be that all of the malloc() calls are done
through a wrapper.  If the code is compiled with an option that adds
additional debugging code, the wrapper adds 8 bytes to the size passed
to malloc(), writes a signature to the beginning of the allocated block
of memory, and adds 8 bytes to the value returned by malloc() before
passing the new value to the caller of the wrapper.  There is also a
wrapper around free() that does the inverse adjustment and sanity checks
the signature.  Even though our malloc() does the proper 16 byte
alignment for memory blocks 16 bytes and larger, the 8 byte offset added
by the wrapper messes this up and we get a SIGBUS whenever the code uses
an instruction sequence that requires 16 byte alignment.
Received on Sat Jan 28 2017 - 08:56:25 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:10 UTC