Wandering Thoughts archives

2008-02-18

Coding paralysis

DWiki's comment system is acceptable as it stands (the proof is in the pudding, in that some people are willing to use it), but it needs to be improved. Specifically, it's been clear to me for a while that there should be 'your name' and 'your website' fields so that people can conveniently identify themselves.

(And so I can have a better idea of who's leaving comments. The current DWiki comment system dates to when it was a half-hearted addition glued on to something I expect to be used primarily as an internal wiki.)

This is a bit of a project, since the comment storage format needs to change (and thus the code needs to deal with comments in either format), but it's not too much work, at least in theory. In practice I have been not doing this for some time; I've wound up in coding paralysis, where I know what I want to do but I can't can't moving on it.

At its heart, my coding paralysis is because I don't feel enthused enough about the changes I want to make; they are feeling too much like work and not enough like fun. At the same time I've thought enough about them that they've become locked in my mind as the 'next step' I want to do, so I don't even think about other DWiki changes that might be more fun.

Probably this really means that I need to find something completely different to code up and obsess over.

Sidebar: some gory details

One reason for my coding paralysis is scope creep; I've let what I want to achieve get too big. For example, I know that I want to add OpenID support someday, and I don't really want to make two significant changes to the comment handling code, so I should really do that as part of the significant revision. But adding OpenID is a bunch of work, and is going to require a bunch of thinking about how to best to add it to DWiki's processing models; it's effectively a significant project on its own, and yet I'm trying to wedge it into my first change.

Another reason is that a chunk of the coding is grunge work, out of proportion to the coolness of the feature it adds. Adding fields to comments requires a new comment storage format and dealing sensible with both old and new comments, which is a bunch of boring code (and I didn't put version information into the comment storage format; bad me).

python/CodingParalysis written at 23:08:20; Add Comment

ZFS versus SANs: where do you put the RAID?

Here is an issue I have been thinking about recently: how do you want to handle ZFS in a SAN environment, especially one in with relatively low data rates such as, oh, anything based on gigabit Ethernet?

The problem is that ZFS and SANs have a small conflict. ZFS wants to talk to raw disks and do RAID internally so that it can do its special magic, but SANs want you to let the backends do as much RAID work as possible to preserve overall bandwidth. If you do mirroring in ZFS, for example, you have just doubled the write bandwidth that you need.

(Read bandwidth should be mostly unaffected, except for your periodic integrity scans. You do do periodic integrity scans, right?)

Various forms of link aggregation and multipathing can give you more iSCSI bandwidth, but my impression so far is that you have to get lucky to have both ends support the same sort of aggregation. It also means a bigger switch infrastructure and at some point you may run into limits on the total bandwidth your switches can support.

How much you have to worry about this depends on how much write bandwidth you need (and on what sort of RAID you'd have ZFS do; mirroring adds more write IO than RAID 6, which has more than RAID 5). If you fall in the middle, with not so little write IO that you can ignore this and not so much write IO that you have no choice, this seems clearly a 'pick your poison' question; both options have disadvantages.

(Here, the winning argument is likely to be that if we let ZFS do the mirroring and always mirror between two SAN backends, our machines may not reboot if they can't talk to one briefly.)

solaris/ZFSvsSANs written at 01:44:18; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.