Re: Apples linking

On Thu, 27 Nov 2003, David Rhodus wrote:

> >FWIW, even though I support the idea of dynamically linking
> >everything, the flipping of the switch there followed this
> >same pattern.
> 
> Terry, what are some of the changes that Apple made to have everything
> dynamically linked in darwin ? Has anyone done timed runs lately on
> dynamically vers. static linking on darwin ? Or did they find just
> cleaning up the dlopen code path seems to be enough to pull dynamically
> linking everything ? 

In Darwin, all binaries but /sbin/init are dynamically linked.  There are
a number of interesting optimizations, including prebinding.  The most
interesting variation on the them is a series of system calls that manage
a special "shared region" for prebound libraries.  These calls are:

int	load_shared_file();
int	reset_shared_file();
int	new_system_shared_region();

Shared regions are managed by privileged processes, such as prebinding
daemons, and are used to hold prebound versions of libraries.  My
understanding is that they are always mapped into processes at the same
address, so a prebound version of the library can be used across many
applications.  In addition, the shared region uses one set of PTEs for all
processes it is mapped into, as well as other VM machinery, so it's very
low cost to maintain.  If a library isn't found in the prebinding cache,
the application does the work itself, but probably sends a message off.
During system/application install, I believe OS X kicks off a tool to see
if its current prebinding cache/layout/etc is optimal for the set of
applications, and adjusts the cache as needed.

I don't have access to any of the performance measurements -- perhaps
Terry does -- but this approach has a number of important benefits.  In
particular, it addresses the following issues: 

(1) Libraries are frequently mapped at different addresses for different
    applications, meaning that prebinding can't be efficient if you're
    just caching the relocation data.  I think I posted some stats in an
    earlier post, but I found even libc was mapped at about 40 different
    addresses on my notebook running XFree86, KDE.  And that is with KDE's
    internal prebinding support.

(2) It avoids substantial overhead in maintaining mappings of files that
    are mapped into many processes.  So not only do you see the benefits
    of prebinding itself, but reduced overhead to many mappings.

(3) Maintains a desirable privilege model, in which applications can't
    mess with the prebinding cache, but can make use of it easily.

Apparently (2) is particular important on PowerPC processors, but
presumably applies to other processor architectures.  The one thing that
turns me off to this scheme is that I'd like it if we could find a way to
represent this using solely existing BSD/UNIX kernel primitives (mmap, et
al) and userspace, rather than adding special-purpose system calls that
complicated various code paths, and that aren't portable.

As I mentioned previously in this thread, it could be we could witness a
lot of the benefits of this approach by simply using heuristics to
increase the likelihood of libraries getting mapped to the same address in
different processes, increasing the effectiveness (and reducing the size) 
of the prebinding cache.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert_at_fledge.watson.org      Network Associates Laboratories