Wandering Thoughts archives

2018-01-10

Linux's glibc monoculture is not a bad thing

Given yesterday's entry about glibc and the Linux API, it's hard to avoid concluding that glibc is basically a monoculture in Linux, with no real competition or alternative. Initially I expected to be unhappy about this since I have a reflexive dislike of monocultures, but after thinking about it more, I think that glibc's monoculture is actually not a bad thing. In fact, I'll go so far as to say that it's how open source should work.

Okay, I confess that I deliberately wrote that to be a bit over the top. So I'll phrase the same situation differently: not having other projects pointlessly trying to compete with glibc is how open source should work (but often doesn't).

If you've been around the open source world for a while, you've seen more than one project that was apparently started because people couldn't just pick one thing to focus all their effort on but had to find some small difference that 'justified' a whole second project with a whole second set of work being done, to very little overall net gain. Over and over again the open source world has N versions of substantial projects that are essentially the same. Oh, sure, there's almost always some reason, but if you take a few steps back and look at the overall pattern, it sure feels like Not Invented Here syndrome is at work at least a fair bit of the time.

(Okay, there's also politics, personalities, and sometimes licensing. Linux has never been an antiseptic and purely technical place, not that the BSDs have been either.)

Given that Linux has several graphical desktops (with libraries and a collection of programs) and religious wars over init systems, there's no obvious reason why we wouldn't also have multiple competing standard C libraries. Instead, we basically have glibc and then a number of other libcs with different goals that are explicitly not trying to be direct replacements or competitors with glibc. For once, almost all of the development effort that's going to a crucial layer of Linux is working on a single project, not being spread over a bunch of NIH competitors.

At the same time, Linux is still open to alternatives and replacements for glibc. For example, the kernel developers haven't said that glibc is the only supported standard C library or ignored backwards compatibility with user space if glibc could paper over the cracks. This is undoubtedly helped by the existence and popularity of real alternatives to glibc like musl and Bionic. If you want (or need) something significantly different to glibc, Linux will still let you get it.

(And things that are intended to be portable outside of Linux will probably accept patches to build with musl and so on.)

As long as glibc development doesn't stagnate or wind up going in bad technical directions, a monoculture around it certainly doesn't seem to be a bad thing. So on the whole I'm happy to see Linux avoiding duplicate work for once.

(At the same time I'll note that glibc is not flawless and yes, I'd like it to be better. Also, in the past there were serious issues with glibc development and some of the people involved in it; see eg here. But back then there was a serious glibc alternative and some Linux distributions actively used it. Both parts of that situation could happen again.)

linux/GlibcMonocultureNotBad written at 23:01:25; Add Comment

Glibc and the Linux API

A while back, I wrote an entry on the overall issue of if the C runtime and standard library were a legitimate part of the Unix API. This was sparked by Go, which tries to avoid the C library on all platforms, Linux included, and where this helped lead to a fun problem. Linux is an unusual Unix in this regard. Many Unixes are tightly coupled to their standard C library, whether or not they officially documented it as the API to the system in general the way Solaris does. OpenBSD, FreeBSD, and so on all have a definitive standard C library that is part of their base operating system along side the kernel (and their standard C compiler); the kernel and this library are developed in sync by the same people. Linux doesn't have an officially blessed or co-developed standard C library in this way; instead it theoretically supports multiple ones. You're supposed to be able to build a Linux system with musl or uClibc-ng or any of the other options out there.

In practice, my understanding is that it's sufficiently hard to replace glibc that no large distribution attempts to do so (although Alpine uses musl and I've heard of it). A certain amount of software doesn't even build when compiled against non-glibc header files, and past that other libc implementations don't necessarily attempt to be exactly compatible (eg, musl's list of differences, and see also). In addition, alternate libcs generally seem to be aimed at being smaller and simpler than glibc, and as part of that they can leave out significant features like nsswitch.conf.

(As a system administrator I'm biased, but yes I consider nsswitch.conf to be a significant feature. It provides significant sysadmin control over a number of important operations.)

Or in short, while Linux officially doesn't have a 'standard C library', in practice no one is attempting to provide a full alternative or replacement to glibc. Glibc is not merely the default choice, it is the only full choice. As a result, whatever API glibc decides to supply for something is the de facto Linux standard. This is especially the case if glibc is putting a wrapper around some new Linux kernel feature. As we saw in marcan's article, there may be undocumented requirements that glibc has reverse engineered but you don't necessarily know.

(It's not clear to me how thoroughly the Linux kernel API is documented, but the man-pages project does appear to cover the actual system calls.)

Also, I believe that there are any number of significant projects that consider glibc to basically be part of the overall Linux API and GNU (libc) extensions to be just as potentially legitimate a standard as anything else. These projects are obviously dependent on glibc (or a feature for feature compatible replacement). In 2014, one such project was systemd, per Lennart Poettering's comments, and I suspect that systemd's views have not shifted since.

(As a practical matter, things like Firefox are not tested, developed, or officially supported against alternate libcs any more than they're tested with unusual C/C++ compilers. If you decide to use them that way anyway, you get to keep all of the many pieces that you're likely to wind up with.)

I have some thoughts on this overall situation, but that's going to have to be another entry since this one is already long and rambling enough.

linux/GlibcAndLinuxAPI written at 02:57:56; Add Comment

By day for January 2018: 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18; before January.

Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.