Wandering Thoughts archives

2011-05-31

The programmer's problem with WikiText systems

Here is a confession: I have been sitting on an update to DWiki that adds some things to its dialect of wikitext for going on four years now. The core problem boils down to two issues specific to wikis.

The first is that in a wiki, additions to your markup language are more or less forever. Once people actually start writing pages that use them, you have three choices; you can support the addition forever, you can drop the addition and live with the resulting broken pages, or you can go through your entire page database and try to rewrite all of the pages to use some new equivalent. It is possible to do the last option well, but people usually don't and then sysadmins hate you.

(If you want to do a good job, reuse your real wikitext parser and just have it emit the correct new version of the wikitext instead of HTML. This way you guarantee that your conversion process parses the old wikitext in exactly the same way that page rendering does.)

The second is the eternal wikitext lament: all of the good formatting characters (and character sequences) are taken. You never have enough formatting markup to go around, and every bit of markup that you use is a bit of plain text that people can't use (this is especially so for the good markup). When combined with the first problem, this means that you want to be really sure that some particular bit of markup is the right use for its particular character sequence before committing to it because you're mostly stuck if it later turns out that you made a bad choice; a bad choice can be really painfully bad, forever locking off an otherwise attractive bit of markup.

It's interesting (well, to me) to think about why HTML doesn't suffer from this issue. Part of it is that HTML adds features slowly, but I think that a large part of it is that HTML's markup is not in short supply the way that wikitext markup is. The goal of good wikitext markup is to be unintrusive and small. HTML doesn't have this concern since it's already made the decision that its markup will be clearly intrusive, and thus it has a much wider range of decent markup to choose from.

(The CS geek way to put this is that HTML has made the decision to put all of its markup in its own namespace, separate from your document's actual text, whereas wikitext markup is trying to live in the same namespace as your document. One of the times that HTML becomes unusually irritating is exactly when the two namespaces can't be kept separate because your document text is riddled with <'s and &'s.)

web/WikiTextDeploymentProblem written at 22:58:31; Add Comment

How to fail at versioning

Today, Ubuntu released a PAM security update and we applied it. On Ubuntu 10.04, this upgraded the Ubuntu PAM package from version 1.1.1-2ubuntu5 to 1.1.1-2ubuntu5.2 (other Ubuntu LTS releases had similarly versions numbers); as you can see, this is a very minor version bump. As you'd expect, this did not change, eg, the libpam shared library soname version (not even the minor version).

Our logs promptly exploded with error messages like:

CRON[12926]: PAM unable to dlopen(/lib/security/pam_env.so): /lib/libpam.so.0: version `LIBPAM_MODUTIL_1.1.3' not found (required by /lib/security/pam_env.so)

(This appears to have affected nearly any daemon or persistent process that uses PAM; cron is just the most obvious one.)

You can make an entire list of ways that this was a versioning fail, and in fact it's a fail on several levels. First, what was labeled as a minor packaging update introduced an ABI incompatibility, one that basically forces a system reboot at that. Second, despite having various versioning mechanisms available to it, PAM made no use of them; for example, it did not bump sonames (not even by a minor number). Finally, PAM claims to have versioning (eg, its library soname is versioned) but it has unversioned components with version dependencies. Here pam_env is clearly versioned; a specific version is ABI-compatible only with a very narrow and specific PAM library. But there is no way to have two pam_env shared objects for two different versions of the PAM library (even if the PAM library made use of version numbering), because it has no version number itself.

(In light of this last issue, it's kind of unsurprising that the libpam soname version did not change; it probably wouldn't do any good even if it did.)

Note that it's not clear who is responsible for all of the failures here. At a minimum Ubuntu is at fault; 'break your system' ABI incompatibilities should never have version number changes that are just minor package updates (not that Ubuntu is all that good at this). If Ubuntu created the ABI incompatibility through one of their patches, they are also at fault for not versioning it properly.

Update: Ubuntu has accepted this as a bug, bug #790538. I suppose the good news is that they consider this a serious issue.

PS: what I can best describe as an extreme reluctance to ever change library soname version despite major ABI changes is not exactly unique to PAM. As far as I can tell it's common behavior for a great many Linux projects, most prominently glibc (which seems to have invented its own additional versioning system because soname versions weren't good enough). I have no idea why people like doing this, although I'm sure there's a reason.

(Possibly changing library soname version numbers on ABI changes was found to not work very well in practice.)

Sidebar: what happens and how to fix it

When this happens to a daemon such as cron or xdm, the daemon basically stops doing much of anything useful; cron did not run cron jobs, and xdm did not let anyone log in. You can cure the problem by restarting the daemon, but note that restarting xdm has the small side effect of immediately terminating the session of everyone who logged in through xdm.

Ultimately you're going to want to reboot the computer. This is kind of troublesome if it is a heavily used login server or a compute server used for long-running jobs. This is still Unix, even though developers seem more and more intent on turning it into 'reboot after doing any changes' Windows.

(Yes, I'm bitter right now.)

linux/PAMVersioningFail written at 01:08:13; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.