Feed aggregators should fail gracefully

December 14, 2008

I don't just mean that they should fail gracefully when the feeds aren't well formed (although that's an important part). There are all sorts of troublesome things that feeds can do; they can be longer than you expected, for example, or they can be sent to you very slowly. In all of these cases, feed aggregators should try to fail gracefully, to extract as much information from the feed as possible unless it is utterly clear that something horrible has gone wrong and you cannot trust the feed at all.

In general all feed readers should do this (or at least all feed readers that are not deliberately being strict to make a point), but I think that it is especially important for feed aggregators to do this. Feed aggregators often have a lot of users behind them, any problems are more likely to be invisible to those users (aggregators are traditionally a lot less transparent than desktop clients, which usually give you some error indications), and even once a user detects that there are problems, they are generally powerless to change settings to fix the situation. The result is that feed aggregators are effectively holding their users hostage to their decisions in a greater way than desktop clients are, and so I maintain that they should be more careful.

(I also think that it is more likely for feed aggregators to have various sorts of limits than desktop clients, simply because an aggregator is probably dealing with more data and more feeds in a situation with less resources.)

This makes me feel that feed aggregators should use a stream oriented parser. Such parsers are less XML-correct (since an XML error at the end of the stream won't be detected until the end of the stream), but are likely to be much better at extracting information from feeds that are incomplete (either naturally or because your aggregator is only willing to look at so much data) or otherwise problematic.

(I admit that my ox is gored on this issue, as the LiveJournal feed of WanderingThoughts once again ran afoul of LiveJournal's arbitrary feed size limit, cutting off updates for about a month until one of my readers left a comment about it here.)

Comments on this page:

By nothings at 2008-12-15 01:22:30:

Btw, in case you were wondering, that post was from me. I couldn't log onto my account on that page; I assume that's intentional since it's in a different part of the hierarchy or something.

By nothings at 2008-12-15 01:24:46:

Also, LJ does allow for changing feed URLs, but there's no automated mechanism for it; you have to submit a support request. So I'm not sure if the support people would actually update it if it seemed to be working currently, but it might be worth a try.

By cks at 2008-12-18 11:22:13:

It's odd that your login wasn't accepted on the about page; it should be, as logins are universal across the entire DWiki.

(Although in testing it now I see that I have a little bug; if you log in on the 'add a comment' page, it should redirect you back to that. Ah, the little details of web applications.)

Written on 14 December 2008.
« The pragmatic problem with strict XHTML validation
Why XHTML is doomed, at least in its strict validation form »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Dec 14 23:24:50 2008
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.