The core of modern Unix shared libraries

September 3, 2011

The fundamental Unix development that enabled modern shared libraries is shared copy-on-write mmap(). That's not necessarily obvious, so let me walk through the logic of this.

There are two problems with shared libraries, with the second stemming from the first. The first problem is how to get them loaded into the process's address space at all, and it has many possible solutions. The easiest solution is actually to have the kernel do it, since after all the kernel is already mapping code and data from the executable; all it needs to do is map some additional things from another file (or several other files).

(Kernel loading was actually how some early shared library implementations worked, but that's another entry.)

The second problem comes from the first problem, and it is how to deal with shared libraries being mapped into different places in memory in different processes while still sharing as much physical memory between processes as possible. Here there are many fewer solutions and most of them are not very good (position independent code has limits, for example). The only really good solution is to fix things up by applying runtime relocations to the shared library, using copy on write to de-share only pages that needed relocations.

In theory the kernel could do this relocation as it loaded each shared library into your process. In practice no one wants to have that much complicated code running in the kernel, so the relocation needs to happen in user space. The simplest way to do that is to have user space handle the entire job of loading shared libraries, and to do that user space needs a way to set up those shared copy-on-write mappings for shared libraries, ie it needs mmap() or something very close to it.

(Doing all shared library loading in user space also allows for all sorts of useful flexibility and powerful features that would be awkward or out of place in a kernel implementation. For that matter, it allows the implementation itself to be replaced.)


Written on 03 September 2011.
« Things I will do differently in the next building power shutdown
How some Unixes did shared libraries in the old days »

These are my WanderingThoughts
(About the blog)

Full index of entries
Recent comments

This is part of CSpace, and is written by ChrisSiebenmann.
Twitter: @thatcks

* * *

Categories: links, linux, programming, python, snark, solaris, spam, sysadmin, tech, unix, web

This is a DWiki.
GettingAround
(Help)

Search:

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Sep 3 00:36:05 2011
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.