How we're planning our backup storage capacity needs
June 5, 2009
Part of the fun of backups is trying to work out how much backup storage you need in order to do a decent job of recovering from mistakes. Unless you are lucky, you will not have a specific mandate about how much to save for how long, which means that it is up to you to figure out a scheme that you can afford, that provides enough coverage, and that people will be happy with; there are no one size fits all answers.
We didn't do anything sophisticated for our old tape based backup system; we let Amanda reuse tapes in its default least-recently-used order, and just bought enough tapes that we could go a year before we cycled around at our planned backup frequency. This worked okay initially but then started falling down when we outgrew our tape backup capacity.
Moving to our disk based backup system has given us an opportunity to rethink our approach, rather than just buying a pile of disks to go along with the pile of tapes we already have. Our current plan is that we will do periodic 'checkpoint archives' to tape (using our old tape backup system and the tapes we already have), probably once every three months, and then have around a month's worth of rolling backups (so that we can go back to any day for up to a month) in the disk based system. Our goal is to have at least a year and ideally two years or more of the checkpoint archives.
One reason for this split scheme is the sort of restore requests we tend to get. When people ask us to restore something from a significant time ago, it's usually some old archival data that they cleaned up thinking that they would never need it again (only to find out otherwise now). As old archival data it tends to have been sitting there unchanged for months (or years), so a backup from anywhere in a broad time range is fine.
Or in short, for old restores we tend to get asked for 'any copy of the file from before time X (when I removed it)'. So we don't need to have lots and lots of copies (one for every full dump in Amanda's regular dump cycle), just a few.
(Disclaimer: as before, I am just writing this down; the hard work and planning was done by my co-workers.)
* * *
Atom feeds are available; see the bottom of most pages.