Re: Warnings about dlclose before thread exit. __cxa_thread_call_dtors

The FreeBSD implementation here looks racy.  If one thread dlcloses an object while another thread is exiting, we can end up calling a function at an invalid memory address.  It also looks as if it may be possible to unload one library, load another at the same address, and end up executing entirely the wrong code, which would have some serious security implications.

The GNU/Linux equivalent of this function locks the DSO in memory until all references to it have gone away.  A call to dlclose() on GNU/Linux will not actually unload the library until all threads with destructors in that library have been unloaded.  I believe that this reuses the same reference counting mechanism that allows the same library to be dlopened and dlclosed multiple times.

It would be nice if the FreeBSD version had the same behaviour, because this is almost certainly expected in code written on other platforms.

David

> On 18 Aug 2018, at 14:18, Willem Jan Withagen <wjw_at_digiware.nl> wrote:
> 
> Hi,
> 
> I've sent the question below to the Ceph-devel list, asking if any recent changes would be able to cause this.
> 
> But then of course this could stem from FreeBSD libs, and of ports....
> So the question here is if anybody has gotten these "warnings" in other tools.
> 
> --WjW
> 
> 
> -------- Forwarded Message --------
> Subject: Warnings about dlclose before thread exit
> Date: Sat, 18 Aug 2018 14:46:35 +0200
> From: Willem Jan Withagen <wjw_at_digiware.nl>
> To: Ceph Development <ceph-devel_at_vger.kernel.org>
> 
> Hi,
> 
> I've have upgraded to FreeBSD ALPHA 12.0, but I don't think the errors them from there. Although they could be in one of the libs that came along with the upgrade.
> 
> I'm getting these warnings during rbd and ceph (maybe even more) invocations that indicate that indicate a possible problem because:
> ===
>   It could be possible that a dynamically loaded library, use
>   thread_local variable but is dlclose()'d before thread exit.  The
>   destructor of this variable will then try to access the address,
>   for calling it but it's unloaded, so it'll crash.  We're using
>   __elf_phdr_match_addr() to detect and prevent such cases and so
>   prevent the crash.
> ===
> this is from : https://github.com/freebsd/freebsd/blob/master/lib/libc/stdlib/cxa_thread_atexit_impl.c
> 
> Now it could be that dlclose() and thread exit are just closed to one another. But still this is hard core embedded in libc already since 2017, so I'm sort of expecting that a recent change has caused this.
> 
> And as indicated it is a possible cause for crashed, because thread_exit is going to clean up things that are no longer there.
> 
> Now the 20 dollar question is:
> 	Where was this introduced??
> 
> Otherwise I'll have to try and throw my best gdb capabilities at it, and try to invoke an rbd call and see where it activates this warning.
> 
> --WjW
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"