2009-05-31
A thought on giving custom redundant storage systems some history
Suppose that you're building some custom storage backend that is simply too big to be backed up, so you have only redundancy; this is probably common if you're building a cloud-style environment or are otherwise dealing with a huge volume of data. This leaves you with the redundancy history problem, where you're protected against hardware failures but any mistakes are 'instantly' replicated to the redundant copies.
Suppose that you want to do better than this; you somehow want to give your redundant storage system some history without going all the way to backups.
The approach that occurs to me is to make your storage system be based around a 'copy on write' model for updates; instead of updating in place, you write new versions and change references (which seems like it would be handy for a distributed system anyways). Then instead of immediately removing unreferenced objects, you try to let them sit around for a certain amount of time (hours, days, or weeks, depending on how much extra storage you can have and what your update volume is).
What this gives you is time. If you make a mistake, you have time to panic, go digging in your datastore, and pull out the now unreferenced objects that correspond to how things used to be. Building tools to help with this ahead of time is probably recommended.
I think that this has two advantages over an actual snapshot feature. First, it has lower overhead in exchange for worse tools to access the 'snapshot' (which is a good tradeoff if you expect to make mistakes only very, very rarely). Second, you aren't restricted to looking only at the points in time where you happened to make snapshots, as you effectively make continuous snapshots.
The better way to install Sun's Java
I've been grumpy for quite a while about how Sun's Java RPM mutilates your system. Since I don't like my systems mangled, I've avoided installing Sun's Java on any of my systems unless I absolutely had to, and if I had to I tried to do it on an expendable test system. This has been vaguely annoying, because even in Fedora 10 there are things that don't work with the substitutes.
(One of them is the KVM over IP portion of the lights out management environment for Sunfire X2100s and X2200s. Since they have a very nice LOM and we've wound up with a bunch, this has been a bit annoying.)
When people offer their software packaged as RPMs, I reflexively pick that option. In the case of Sun, it turns out that this is a mistake, because the non-RPM version of Sun Java is much better. It is happy to be installed by a non-root user, in which case it doesn't mangle your system at all; instead it drops itself into a single directory hierarchy (and you can even move it around).
(It helps to know that the 'self-extracting file' seems to extract itself into your current directory.)
Right now I'm just interested in dealing with is 'JNLP' files (Java
Webstart executables, which are run by javaws, which may be present
only in the 32-bit version of Java), and for them I didn't even need to
add things to my $PATH; I just pointed Firefox at the right executable
once, and I was done.
All in all, I feel somewhat silly for not having looked at the non-RPM version a long time ago. If I had, I could have saved myself years of low-level irritation.
PS: according to here you may need some compat RPMs (I had them installed already).