Wandering Thoughts archives


Feed aggregators should fail gracefully

I don't just mean that they should fail gracefully when the feeds aren't well formed (although that's an important part). There are all sorts of troublesome things that feeds can do; they can be longer than you expected, for example, or they can be sent to you very slowly. In all of these cases, feed aggregators should try to fail gracefully, to extract as much information from the feed as possible unless it is utterly clear that something horrible has gone wrong and you cannot trust the feed at all.

In general all feed readers should do this (or at least all feed readers that are not deliberately being strict to make a point), but I think that it is especially important for feed aggregators to do this. Feed aggregators often have a lot of users behind them, any problems are more likely to be invisible to those users (aggregators are traditionally a lot less transparent than desktop clients, which usually give you some error indications), and even once a user detects that there are problems, they are generally powerless to change settings to fix the situation. The result is that feed aggregators are effectively holding their users hostage to their decisions in a greater way than desktop clients are, and so I maintain that they should be more careful.

(I also think that it is more likely for feed aggregators to have various sorts of limits than desktop clients, simply because an aggregator is probably dealing with more data and more feeds in a situation with less resources.)

This makes me feel that feed aggregators should use a stream oriented parser. Such parsers are less XML-correct (since an XML error at the end of the stream won't be detected until the end of the stream), but are likely to be much better at extracting information from feeds that are incomplete (either naturally or because your aggregator is only willing to look at so much data) or otherwise problematic.

(I admit that my ox is gored on this issue, as the LiveJournal feed of WanderingThoughts once again ran afoul of LiveJournal's arbitrary feed size limit, cutting off updates for about a month until one of my readers left a comment about it here.)

tech/GracefulAggregatorFailure written at 23:24:50; Add Comment

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.