Wandering Thoughts archives

Side effects are bad: a personal illustration

One of the things they tell you in programming school is 'side effects are bad; you should avoid them if possible'. Apparently every now and then I need a reminder of this, because side effects were at the bottom of my feed whoops from last night.

One of the most expensive bits of DWiki is converting pages from DWikiText into HTML. Originally the conversion was simple and straightforward; stick wikitext in, get HTML out. But as DWiki grew more features, the wikitext started having additional information: whether people could see it, whether people could comment on the page, and so on. These days converting a page to HTML generates all of:

• the HTML
• whether the page restricts access and/or allows comments
• various fuzzily defined features of the page
• the title of the page, if any

The conversion routine returned the HTML and recorded everything else by manipulating a per-request context structure; in other words, through side effects.

One of the problems with side effects is that you forget them, which is just what I did when I added caching of the conversion process. I remembered the other two side effects and cached them (partly because I'd been working in the area recently), but forgot about page titles. When the cached data was restored, page titles weren't set and things reverted to default titles in feed entries.

(One of the reasons I forgot about the page titles is the other problem with side effects, namely that they're usually scattered over the code instead of centralized in one spot. If the renderer had returned all of the results in one structure and broken them up later, I might not have wound up in the pickle I did.)

Update, April 1st: I just discovered another side effect I had forgotten; updating the timestamp for the request's Last-Modified: header. Oh the irony; apparently I like playing April Fools' jokes on myself.

We apologize for the disruption in syndication feeds

In the process of enabling caching in DWiki, I managed to illustrate the importance of thorough testing by failing to properly cache the real titles of entries and then not noticing, because that the Atom feeds stayed the same was the one thing I didn't test. (I carefully and somewhat obsessively tested that the rendering of all of the other pages did stay the same. I even have that automated.)

This caused the syndication feeds for WanderingThoughts to revert to plain filename based titles for eight hours or so, until I noticed (yet another example of ReadYourFeed; I did eventually, but not fast enough). So everyone just saw a gratuitous feed change or two, which I apologize for.

The effects on the LiveJournal feed were a little bit more dramatic. Despite the only change to the feed being the titles, LiveJournal decided to revive two entries from March 7th and 8th and arbitrarily give them today's date, effectively regurgitating stale content on the readers of the LiveJournal feed. This irritates me, but probably not as much as it irritates you.

(I call it arbitrarily because the entries have proper (old) timestamps in the Atom feed, and they're not at the top of the Atom feed either. LiveJournal's syndication stuff mystifies me on a regular basis. Hopefully it is not doing something as crazy as using the entry title as its internal identifier.)

The incident has also pointed out some problems in my caching implementation. In particular, I need to version the cached data so that a new DWiki version using a cache with old data silently ignores it instead of exploding spectacularly. (Alternately I need a more robust way of storing the data, because sooner or later I may forget to bump the data version number during an upgrade and then have the kabooms anyways.)

Caches are hard. Let's go shopping for faster CPUs.

Atom versus RSS

David Heinemeier Hansson in the comments on one of his entries:

Joe: Atom is just RSS without the bugs. [...]

What he said.

The more I've learned about syndication formats, the more thankful I've been that I picked Atom way back when. (I'm not sure why I chose Atom; possibly because it seemed the more up to date of the choices at the time, since it had an RFC in development and all.)

The difference between Atom and RSS is that Atom has a real specification, one good enough that people actually write to it. So I can use the specification to write a useful feed generator, which is pretty much what I did for DWiki (with some help from the feed validator).

With RSS, the formal spec is unclear and incomplete, so in practice RSS is defined by what popular feeds and feed readers do. This had led to various problems and dark corners, where sometimes nothing you can do is going to work for everyone. (I'll stop footnoting that now; I could go on all day.)

In my opinion, voodoo programming is no way to run a railroad. So I am really glad I went with Atom; I would probably have found dealing with all of the RSS issues to be teeth-grindingly frustrating.

(The ongoing RSS soap opera doesn't help either, but mostly it makes me glad I am nowhere near the blast radius.)

Solaris patch exit codes and what they mean

This isn't my work; it's from a comment on an earlier entry. I'm repeating it here partly to have it handy, and partly because DWiki mangled the nice clear plain ASCII formatting of the original. So thank you, mostly anonymous commenter.

The following are the explanation of patchdd script exit codes:

Why this information is not in the CLUSTER_README I don't know. It appears to be in some patch cluster READMEs, but not the Solaris 9 recommended patch set one. Yet another Sun mystery, I suppose.

(PS: the formatting on this may not look so great in syndication feeds, including LiveJournal. Explaining the mess of why requires a larger rant than this margin has space for.)