Wandering Thoughts archives

OmniOS (and by extension Illumos) is pretty much Solaris

I've been working on bringing our customized Solaris fileserver environment up on OmniOS, which is going to power the new generation of our fileservers. On our current Solaris fileservers we do things that range from relatively standard through fairly crazy, so one of my concerns with OmniOS was how easy these things would be to port over to it (especially the fairly crazy things).

I've been pleasantly surprised by the answer, which is that for the most part OmniOS is Solaris. In fact it is so much Solaris that for a while I was using our Solaris custom NSS shared library instead of bothering to recompile it on OmniOS (I did eventually because I wanted to be sure that we could still build it and have it work right on OmniOS; it seemed safer). This is what one would hope for, of course, but given my Linux experience and that there are changes from Solaris to Illumos (and OmniOS) it's still pleasant to see it happen.

The major of the changes I've needed to make have been for two reasons. First, the paths to various additional bits of software have changed between Solaris and OmniOS. Partly this is because OpenCSW doesn't do Illumos builds and we switched to pkgsrc, partly it's because some things moved into OmniOS's core (eg rsync, which I applaud), and partly it's because things just moved period (eg a bunch of things on OmniOS hang out in /usr/gnu/bin).

Second, parts of our spares system fish around in system-dependent areas of ZFS in order to read out detailed information on pool state. There's still no stable public interface for this information and the format of stuff changed between Solaris 10 update 8 and the current version of OmniOS. With full OS source available this wasn't particularly difficult to deal with, and anyways we knew that this was coming (it was a big reason we never moved beyond S10U8).

(A minor change was that a bunch of our management tools used to assume that the only ZFS pools on the system were our pools. This is no longer true on OmniOS, where you also have the root ZFS pool. We want some of our tools to pay attention to the rpool, for example to scrub it regularly and tell us if it has problems, but we want others to ignore it, eg the spares system. We may be fiddling with this for a bit.)

PS: The whole OmniOS build is not completely tested and deployed into a test production fileserver, so I may yet find another significant source of changes. And our DTrace scripts will probably need some changes and updates because they play around in kernel data structures, some of which have changed between S10U8 and OmniOS. For that matter, DTrace itself has probably evolved since S10U8 and it may be worth revising some scripts to take advantage of that.

The problem with OmniOS's 'build your own' view for Perl and Python

A commentator on my entry on Python 2's lifetime mentioned that OmniOS has adopted a principle called KYSTY. To simplify, the core of this principle is that you are not really supposed to use the OmniOS versions of things for your applications. To quote them:

We purposefully ship only what we need to build and run the OS. In many cases, these are not the most recent versions. Except in the most basic circumstance, you should not use these things in your app stack. At best, their versions are stagnant; at worst, they may go away entirely as the OS components they exist for are rewritten or removed.

There are two problems with this when it comes to things like Perl and Python. The first is a variant of the problem with compiling your own version of Python 3; requiring people to build or get their own versions of Perl and Python makes writing small utilities in those more expensive. In practice many people aren't going to do that and will instead use the system versions, with various consequences.

The larger problem is what this does to script portability in heterogenous environments. Like it or not, there is a de facto Unix standard about where to find at least Perl and Python and that is at well known names in /usr/bin. Making those names not work right in favour of, say, /opt/csw/bin/perl means that scripts that work on OmniOS won't work on other machines and vice versa; each side's scripts will have a #! line that points to something that is either wrong or not there at all. This is not doing people any favours (and yes, some people still run heterogenous environments with shared scripts).

(The #!/usr/bin/env hack doesn't help because you can't count on optional add-on directories being in $PATH in all cases and all contexts. In fact I tend to think that it is a great recipe for real problems but that's another rant.)

The OmniOS approach is arguably sensible for large applications with their own deployment infrastructure and workable if OmniOS is by itself in a homogenous environment. It is not appealing if OmniOS is in a heterogenous environment and what you're writing is glue scripts and utilities and so on.

If OmniOS wants to take KYSTY seriously it needs to not claim the names /usr/bin/perl, /usr/bin/python, and so on. OmniOS Perl and Python scripts should use a private OmniOS name for the OmniOS versions of those and leave the official public names to sysadmins to set up however they want (or not set up at all, so that you know that all of your scripts have to be using your specific version of whatever).