One of the things that I've been doing over the last while is thinking about what our long-term future with Solaris is; my best guess is that 'long term' here starts somewhere around 2013. In theory the simple thing to do is to keep running Solaris in basically the same setup as we have now, just on updated hardware, but there are two potential problems with that idea.

The first is that it isn't at all clear if there will be affordable inexpensive 1U or 2U servers that we can legally run Solaris on, and even if there are we don't know if we'll be able to afford Oracle's support rates (or at least if we can justify spending that much money a year on it). In theory there might be officially supported inexpensive 1U servers from HP or Dell, but given Oracle's back and forth reactions to Solaris on third-party hardware, I'm not entirely sure I would trust doing that; even if Oracle supports it this year, what happens if they change their minds again?

(This is one result of Oracle screwing people.)

The second potential problem is that a fair bit of what we do with our fileserver environment today relies on having (Open)Solaris source code available to study, in order to figure out how to talk to undocumented libraries and interpret undocumented data structures. OpenSolaris source code is not being updated right now, and it remains to be seen if it ever will be or how frequently. Running Solaris without source is significantly more risky for us (one way or another), unless Oracle officially exposes a lot more interfaces and information in Solaris 11. For example, our alternative ZFS spares system (which we wrote for good reasons) relies on the ability to directly see ZFS pool state information, including information that is not exposed by 'zpool status' (even if I wanted to parse its output), and I don't think we'd be very happy going back to the old ZFS spares system (even if it was bug-free this time around).

(Despite what I've said before, I don't think that Solaris 11 is going to go in directions that we don't want. If anything, I'm sort of looking forward to changes like a real package system, and overall I expect it to be better than Solaris 10.)