Re: Apples linking

David Rhodus wrote:
> Terry Lambert wrote:
> >FWIW, even though I support the idea of dynamically linking
> >everything, the flipping of the switch there followed this
> >same pattern.


First, a disclaimer: this is me speaking for me; I do not speak
for Apple.


> Terry, what are some of the changes that Apple made to have everything
> dynamically linked in darwin ?

I can't speak authoritatively, since I wasn't ther when it happened;
however, there are two types of dynamic linking in MacOS X, and that's
probably the major thing.

The first kind of dynamic linking is more or less the traditional
sort, where you have a single library image, and binding occurs
at runtime.

The second kind uses what's called a "split image", where code and
data sections are split and relocated at offsets relative to the
addresses 0x80000000 and 0x90000000.  These addresses were chosen
because, like FreeBSD, the Intel version of Darwin maps process
memory into the kernel address space.  I expect that Intel Darwin
is going to have to separate the kernel map from the user map, and
thus free up 3G of address space for the kernel and 1G of address
space for user processes, at some point, but I have no idea if there
are plans for this or not, it's just that I think it would be a good
idea.  The 1:3 relationship between KVA and UVA in Intel Darwin is
problematic for large memory machines, for obvious reasons, and
dictates the location of the slit library map and the location of
the "commpage" (introduced in the most recent Darwin release).  The
library code is loaded via a load_shared_file() system call.  The
management of this space is given over to Apple's B&I: third parties
can't create new split libraries; split libraries are limited to
system libraries supplied by Apple.

In the "split image" case, the libraries are prebound in the
executable images because they are at known offsets.  The code is
mapped directly, and the data is also mapped directly, but mapped
as copy on write.  In this case, the libraries most resemble the
System V shared library model, with no support for versioning, and
the binding occurs at installation time, and may occur again at
runtime, should conditions change.  The same map area is used for
both code and data (there's no good reason for this), meaning the
reserved space is max(sizeof(code),sizeof(data)), plus some fudge
so that the library offsets need not be recalculated for every new
instance.

Obviously, this prebinding is much more explicit, and does not
suffer the runtime penalty for most of the shared libraries in OS's
like FreeBSD or Linux, it's not like the prebinding in DragonFlyBSD
(which has a higher runtime penalty than the Darwin approach).


> Has anyone done timed runs lately on dynamically vers. static linking
> on darwin ?

Yes.  It's my understanding that this is why prebinding exists.
Again, this predates me, so my understanding here may be flawed.


> Or did they find just cleaning up the dlopen code path seems
> to be enough to pull dynamically linking everything ?

Dynamic linking in Darwin doesn't use dlopen, it uses another
family of routines.  The dlopen code didn't exist in Darwin until
the latest release (Darwin uses dyld internally); in the latest
release, there is a compatability wrapper for dyld that implements
dlopen and family as conformant interfaces, but it's important to
note that these are library wrappers for library wrappers for
system calls, and are implemented indirectly; it's somewhat faster
to use the dyld code directly, if you have speed requirements.

Neither of these are used for shared library linking, for split
shared libraries, so the question isn't meaningful to ask in the
context of shared libraries.

FWIW: If FreeBSD wanted to use this approach, the safest way to do
it would b to split the user and kernel address space mappings; in
general, this will only mean modifying uiomove/copy{in|out}[str],
and dealing with the address mapping itself.  This would free up
the KVA space not available to user space for use in mapping the
split shared libraries.

-- Terry