Wandering Thoughts archives

What everyone needs in source packages

If you have a packaging system for open source programs (and who doesn't these days) and you have the concept of 'source packages' as distinct from the compiled binary packages (which you probably do), there are four things that source packages need:

• metadata about the package (or packages), such as the version and description.

• the original 'upstream' source code (or some proxy for it).

• whatever changes and additions you make to the upstream source code, if there are any (and there usually are).

• instructions on how to compile the code and package it.

(If you are building your own code you can generally leave out the 'local changes' stuff, but you still need the the metadata, the source code, and the build instructions.)

In theory if you are sufficiently crazy you can merge together the middle two pieces, so that the source code in your source packages is pre-modified and ready to compile. People who do this generally find out how bad an idea it was when they want to start updating their package builds to the next upstream release.

(I'm not sure if any actual packaging system has made this mistake, but plenty of people certainly made it on their own in the days before formal packaging systems when you plunked stuff in a local directory, hacked it up to work, and then did 'make && make install'.)

Actual packaging systems vary tremendously in how they handle and represent all four of these things. For example, I understand that the packaging system used by the *BSDs generally doesn't even store the original upstream source code in the source package and just includes information on how to fetch it at build time.

(While I think that there are better and worse approaches to handling all four things, I don't think that there's a clear right answer as to how to do it. And to a certain extent the best approach depends on how your system is likely to be used; the obvious case is something designed to package your own source code is likely to look quite different from a system aimed at packaging other people's code.)

What makes an operating system attractive

A commentator on yesterday's entry wrote:

The feature set of Solaris 10, which I thought was quite outstanding from an OS point of view at the time, never seemed to gain much traction with decision-makers (possibly because they tend not to have an "OS point of view").

Over time I've come around to believing two general things about operating systems (partly as a reaction to dealing with Solaris and reflecting on how I feel about it).

First, operating systems are commodities. Not in the sense that they're easily substituted for each other, but in the sense that no one is very interested in an operating system by itself. What matters is what you do on top of the OS, not the OS (a computer with an OS but no applications is fundamentally an expensive doorstop). What you could call the OS point of view only matters to a certain sort of technical person.

(This was a hard lesson to learn because I'm the sort of technical person who does care about the OS. At least to some degree, and finding the limits of that caring was sort of humbling when it happened.)

Second, most systems work almost all of the time. Most people and places don't run or deploy unstable systems and are not pushing the limits of what their systems can do. In fact people generally prefer to stay well back from the edge (sometimes to the despair of technical people when management picks the nominally tried and true safe choice over others). One consequence of this is that fault tolerance and fault diagnosis features are generally not all that attractive to people, because most people never expect to need them. Another consequence is that what actually matters is how the system works in day to day operation, because that's most of how people will interact with it.

For server applications, Unix OSes are in a peculiar situation because from the commodity viewpoint they actually approach being a true commodity. Many programs will run on top of most or any of them, so your choice of which Unix OS is often not forced by the higher-level software you want to use; instead you can make it based on other criteria.

All of this leads me to a two-part view of OS attractiveness, especially for Unixes. To technical people in most environments, what matters is how easy it is to get the software you want set up and how much of a pain it is to run the system in routine use, especially in day to day operation. To decision makers what matters is partly what their technical people want to use, partly what the software (and its hardware) costs, and partly whether the OS is seen as a safe, sensible choice.

Actual technical quality doesn't really come in to this, certainly not directly (not once you have a level of basic competence). Technical quality may well affect other aspects that do matter but by itself it's not sufficient; a technically solid OS can easily fumble the higher level issues, for example suffering from being costly, unpopular, and a pain to administer.

(This is not irrational decision making on the part of management, either. Management's job is to deliver results, not to love technology. (Actually that's everyone's job, but that's another topic.))