Re: svn commit: r339898 - head/lib/libc/amd64/sys

On Mon, Nov 05, 2018 at 09:10:13PM -0500, Charlie Li wrote:
> On 03/11/2018 19:45, Konstantin Belousov wrote:
> > Or rather, it is a middle of the valid instruction.
> > Next frame looks like it is process_irelocs(), if trusting the line
> > numbers.  So most likely it is something related to calling wrong
> > relocator function, if anything.
> > 
> > Perhaps you could try to trace the things manually, doing
> > single-stepping of the startup code in debugger. There should be very
> > modest amount of the irelocs, perhaps only one, and see where things go
> > off the way.
> > 
> After a few more complete buildworlds, including one with all
> bootstrapping enabled, this doesn't look compiler-specific. Static
> binaries built with the in-tree base LLVM (6.0.1) also crash.
For you, but not for me.

> 
> I stepped through bmake with base lldb, comparing the working copy in my
> system from circa r339990 with r340173 built with bootstrapped
> toolchain. Only the differing parts are shown for conciseness.
> 
> Circa r339990:
> Process 82271 stopped
> * thread #1, name = 'make', stop reason = step over
>     frame #0: 0x000000000024ab06 make`_init_tls at tls.c:471
>    468          }
>    469          tls = _rtld_allocate_tls(NULL, TLS_TCB_SIZE, TLS_TCB_ALIGN);
>    470
> -> 471          _set_tp(tls);
>    472  #endif
>    473  }
> (lldb) n
> Process 82271 stopped
> * thread #1, name = 'make', stop reason = step over
>     frame #0: 0x0000000000255e60 make`_set_tp(tp=0x00000008002f7830) at
> _set_tp.c:38
>    35
>    36   void
>    37   _set_tp(void *tp)
> -> 38   {
>    39
>    40           amd64_set_fsbase(tp);
>    41   }
> (lldb) s
> Process 82271 stopped
> * thread #1, name = 'make', stop reason = step in
>     frame #0: 0x0000000000255e64 make`_set_tp(tp=0x00000008002f7830) at
> _set_tp.c:40
>    37   _set_tp(void *tp)
>    38   {
>    39
> -> 40           amd64_set_fsbase(tp);
>    41   }
> (lldb) s
> Process 82271 stopped
> * thread #1, name = 'make', stop reason = step in
>     frame #0: 0x0000000000256580
> make`amd64_set_fsbase(addr=0x00000008002f7830) at amd64_set_fsbase.c:43
>    40   #include <machine/specialreg.h>
>    41   #include <machine/sysarch.h>
>    42   #include <x86/ifunc.h>
> -> 43   #include "libc_private.h"
>    44
>    45   static int
>    46   amd64_set_fsbase_cpu(void *addr)
> (lldb)
> 
> r340173:
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = step over
>     frame #0: 0x0000000000247c96 make`_init_tls at tls.c:471
>    468          }
>    469          tls = _rtld_allocate_tls(NULL, TLS_TCB_SIZE, TLS_TCB_ALIGN);
>    470
> -> 471          _set_tp(tls);
>    472  #endif
>    473  }
> (lldb) n
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = step over
>     frame #0: 0x0000000000252eb0 make`_set_tp(tp=0x00000008002ed830) at
> _set_tp.c:38
>    35
>    36   void
>    37   _set_tp(void *tp)
> -> 38   {
>    39
>    40           amd64_set_fsbase(tp);
>    41   }
> (lldb) s
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = step in
>     frame #0: 0x0000000000252eb4 make`_set_tp(tp=0x00000008002ed830) at
> _set_tp.c:40
>    37   _set_tp(void *tp)
>    38   {
>    39
> -> 40           amd64_set_fsbase(tp);
>    41   }
> (lldb) s
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = step in
>     frame #0: 0x0000000000252eb5 make`_set_tp(tp=0x00000008002ed830) at
> _set_tp.c:40
>    37   _set_tp(void *tp)
>    38   {
>    39
> -> 40           amd64_set_fsbase(tp);
>    41   }
> (lldb) s
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = step in
>     frame #0: 0x00000000002ebdb0 make
> ->  0x2ebdb0: movq   0x3ce9(%rip), %r11
>     0x2ebdb7: callq  0x2ebda0                  ; _fini
>     0x2ebdbc: jmp    0x2ebd92                  ; _init + 6
>     0x2ebdc1: pushq  $0x0
> (lldb) n
> Process 22663 stopped
> * thread #1, name = 'make', stop reason = instruction step over
>     frame #0: 0x00000000002ebdb7 make
> ->  0x2ebdb7: callq  0x2ebda0                  ; _fini
I guess this is where things go off for you, but I may be wrong.
After ifuncification, 'amd64_set_fsbase()' line should be represented
by the asm as either
	call	<somewhere in plt>
and the place in plt is just
	jmp 	*(rip-based offset to GOT)(%rip)
In fact the call to amd64_set_fsbase() in the tail-call position, so
the first call is jmp.

For me, everything works.  If for you it does not you should look at the
instructions and see which values went off.  You completely omitted that
details from your trace, so I cannot even guess which part was corrupted.
Again, for me it works with the in-tree toolchain, so I am quite sure that
you have trouble with the toolchain.