2014-08-04
Why our new SAN environment is separate from our old SAN environment
Our move from our old ZFS fileservers to our new ZFS fileservers is a total move; we're replacing both the fileservers and the backends and migrating all of the data to newly created ZFS pools (primarily because of the ZFS 4K sector disk issue). However, there are at least two ways to do this overall move.
The first way is to build the new environment out so that it is completely separated from the old environment and both run independently. The new fileservers get new virtual fileserver names and migrating a filesystem from the old to the new setup changes its NFS fileserver (which is slightly user visible) and requires various administrative changes, eg to our backup system. This is the straightforward and obvious approach.
ZFS gives us the option of a second, perhaps more clever way. First, connect up the new fileservers and backends to the existing iSCSI fabric. Then we can have the new fileservers take over from existing fileservers, importing the existing ZFS pools from the existing iSCSI backends and using the same virtual fileserver names; from the outside it would be as if each fileserver had a (slow) reboot. Once the old pools were up and running, we could create new pools on the new storage and copy filesystems across (unfortunately the NFS clients would probably have to unmount and remount the filesystem during the migration).
(We would need to finess some issues, but it's doable.)
I'll end any suspense here: we're taking the first approach. The new fileserver environment isn't connected in any way to the old one (in specific, the iSCSI networks are completely disconnected from each other). This isn't a decision that we directly considered, but I think that was because we all thought it was so obvious a choice that we didn't even need to talk about it. A fully independent new environment might or might not be somewhat more work, but it is undeniably less risky. We don't have to worry that the new environment will perturb the functioning of the old one or vice versa, because they can't.
(I admit that part of this is technical because it's not clear how we'd interconnect the 10G and 1G iSCSI networks so that fileservers on the 10G network had enough bandwidth to talk to the 1G iSCSI backends at decent aggregate speeds (our existing 1G iSCSI switches don't have SFP+ ports). If there's not enough bandwidth for decent performance, we might as well not interconnect them and then we're in the 'fully independent' setup.)