Re: Apples linking

Robert Watson wrote:
> On Thu, 27 Nov 2003, David Rhodus wrote:
> > what are some of the changes that Apple made to have everything
> > dynamically linked in darwin ? Has anyone done timed runs lately on
> > dynamically vers. static linking on darwin ? Or did they find just
> > cleaning up the dlopen code path seems to be enough to pull dynamically
> > linking everything ?
> 
> In Darwin, all binaries but /sbin/init are dynamically linked.

There are actually a couple more than that.  I specifically did an
audit of which programs on a fully installed binary Panther system
were statically vs. dynamically linked.  8-).

> There are
> a number of interesting optimizations, including prebinding.  The most
> interesting variation on the them is a series of system calls that manage
> a special "shared region" for prebound libraries.  These calls are:
> 
> int     load_shared_file();
> int     reset_shared_file();
> int     new_system_shared_region();

Correct.  You should also read this as "never use these calls in
your own code".

> My
> understanding is that they are always mapped into processes at the same
> address, so a prebound version of the library can be used across many
> applications.

Yes.  Their location is arrived at dynamically, and fixed statically;
this has the same advantage as System V shared libraries, but doesn't
burn chunks of address space for different versions of the same thing,
as time goes forward.  In particular, the libraries can be moved around,
but when they are, the code that depends on them is re-prebound to know
about the new location.

> If a library isn't found in the prebinding cache,
> the application does the work itself, but probably sends a message off.

It does.  This message triggers a prebinding.  Normally, this is
done automatically at the time the applicaiton is installed, but
there are circumstances where it needs to be triggered lazily, and
this handles that case.


> During system/application install, I believe OS X kicks off a tool to see
> if its current prebinding cache/layout/etc is optimal for the set of
> applications, and adjusts the cache as needed.

If you use the standard installation tools ("ViSE"), yes.


> I don't have access to any of the performance measurements -- perhaps
> Terry does -- but this approach has a number of important benefits.

I do, but the information is proprietary, for obvious reasons.

> In
> particular, it addresses the following issues:
[ ... ]

(4) It deals with the issue of needing fixup tables for shared
    libraries to get the right address, so it saves the fixup
    and the indirect that would be otherwise needed, as well.

> The one thing that
> turns me off to this scheme is that I'd like it if we could find a way to
> represent this using solely existing BSD/UNIX kernel primitives (mmap, et
> al) and userspace, rather than adding special-purpose system calls that
> complicated various code paths, and that aren't portable.

It would b relatively trivial to add two flags, on to mmap(), and
one to madvise().  The mmap() flag would indicate a global mapping
into all process address spaces (and select a mapping in a reserved
memory range for that purpose).  The madvise() flag would indicate
that the prebind region was being recreated for use by subsequently
started processes.


> As I mentioned previously in this thread, it could be we could witness a
> lot of the benefits of this approach by simply using heuristics to
> increase the likelihood of libraries getting mapped to the same address in
> different processes, increasing the effectiveness (and reducing the size)
> of the prebinding cache.

This is actually unlikely.  Minimally, you would need to ensure
against collisions, which is hard nough, but then you must also
avoid collisions with shared memory/mmap/dlopen objects, as well.
For this to work, a reserved address region is required.  And for
that, you would ned to take it from the KVA by splitting out the
address spaces so that UVA = 4G, KVA = 4G instead of UVA + KVA = 4G,
as it is today.

This is something FreeBSD should consider anyway: a reserved space
could be very useful, if only to represent the proc struct as R/O
in user space to avoid all system call overhead for getpid/getppid/etc.
system calls.  It would also be useful to avoid reducing UVA to 2G or
1G (depending on intended memory usage patterns) when you have 4G of
physical RAM installed in a machine.  It would also give enough room
so that the kmem_map would not need careful tuning to avoid mmory
exhaustion panics.

-- Terry