Some exciting ZFS features that are in OmniOS CE's (near) future

March 19, 2018

I recently wrote about how much better ZFS pool recovery is coming, which reported on Pavel Zakharov's Turbocharging ZFS Data Recovery. In that, Zakharov said that the first OS to get it would likely be OmniOS CE, although he didn't have a timeline. Since I just did some research on this, let's run down some exciting ZFS features that are almost certainly in OmniOS CE's near future, and where they are.

There are two big ZFS features from Delphix that have recently landed in the main Illumos tree, and a third somewhat smaller one:

  • This better ZFS pool recovery, which landed as a series of changes culminating in issue 9075 in February or so. Although I can't be sure, I believe that a recovered pool is fully compatible with older ZFS versions, although for major damage you're going to be copying data out of the pool to a new one.

  • The long awaited feature of shrinking ZFS pools by removing vdevs, which landed as issue 7614 in January. Using this will add a permanent feature flag to your pool that makes it fully incompatible with older ZFS versions.

  • A feature for checkpointing the overall ZFS pool state before you do potentially dangerous operations and can then rewind to them, issue 9166, which landed just a few days ago. Since one of the purposes of better ZFS pool recovery is to provide (better) recovery over pool configuration changes, I suspect that this new pool checkpointing helps out with it. This makes your pool relatively incompatible with older ZFS versions while a checkpoint exist.

(Apparently Delphix was only able to push these upstream from their own code base to Illumos and OpenZFS relatively recently.)

All of these features have been pulled into the 'master' branch of the OmniOS CE repository from the main Illumos repo where they landed. Unless something unusual happens, I would expect them all to be included in the next release of OmniOS CE, which their release schedule says is to be r151026, expected some time this May. This will not be an LTS release; if you want to wait for an LTS release to have these features, you're waiting until next year. Given the likely magnitude of these changes and the relatively near future release of r151026, I wouldn't expect OmniOS CE to include these in a future update to the current r151024 or especially r151022 LTS.

Since OmniOS CE Bloody integrates kernel and user updates on a regular basis, I suspect that it already has many of these features and will pick up the most recent one very soon. If this is so, it gives OmniOS people a clear path if they need to recover a damaged pool; you can boot a Bloody install media or otherwise temporarily run it, repair or import the pool, possibly copying the data to another pool, and then probably revert back to running your current OmniOS with the recovered pool.

Sidebar: How to determine this sort of stuff

The most convenient way is to look at the the git log for commits that involve, say, usr/src/uts/common/fs/zfs, the kernel ZFS code, in the OmniOS CE repo. In the Github interface, this is drilling down to that directory and then picking the 'History' option; a convenient link for this for the 'master' branch is here.

Each OmniOS CE release gets its own branch in the repo, named in the obvious way, and each branch thus has its own commit history for ZFS. Here is the version for r151024. Usefully, Github forms the URLs for these things in a very predictable way, making it very easy to hand-write your own URLs for specific things (eg, the same for r151022 LTS, which shows only a few recent ZFS changes).

There's no branch for OmniOS CE Bloody, so I believe that it's simply built from the 'master' branch. It is the bleeding edge version, after all.


Comments on this page:

By Oma at 2018-04-04 08:15:42:

I would like to see in illumos the improved n-way mirroring... FreeBSD and Linux have it already. http://open-zfs.org/wiki/Features#Improve_N-way_mirror_read_performance

Let's keep hoping.

Written on 19 March 2018.
« Why I use Facebook (a story of web development)
Python and the 'bags of unstructured data' approach »

Page tools: View Source, View Normal.
Search:
Login: Password:

Last modified: Mon Mar 19 01:24:50 2018
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.