ZFS's potentially very useful '
zpool history -i' option
I recently wrote a little thing praising
zpool history. At the time I wrote that, I hadn't really
read the manpage carefully enough to have noticed an important
additional feature, which is
-i argument (and
-l as well, sometimes). To quote the manpage,
internally logged ZFS events in addition to user initiated events'.
What this means in plain language is that '
zpool history -i' shows
you a lot of what happened to your pool no matter how it was done.
This may sound irrelevant and abstract, so let me give you a concrete
Did you know that you can create and delete snapshots in a filesystem
rmdir in the <filesystem>/.zfs/snapshot
directory? If you have sufficient privileges (root is normally
required), this works both locally and over NFS to a ZFS fileserver. Snapshots created and deleted this way don't
show up in plain '
zpool history' because of course they weren't
created with a '
zfs' command, but they do show up in '
When you're looking at the output at this level, you will typically see three log events for a typical command:
<time> [txg:12227245] snapshot fs0-core-01/cs/mail@2017_01_10 (4350)<time> ioctl snapshot input: snaps: fs0-core-01/cs/mail@2017_01_10 props: <time> zfs snapshot fs0-core-01/cs/mail@2017_01_10
[txg:NNN] first line is the low-level internal log and is
apparently the only log entry that's guaranteed to be there, I
assume because it's written as part of the transaction; the remaining
records can be lost if the machine fails at the right time or the
program crashes, and they're written after the TXG record (as we
see here). The
ioctl entry tells us that this was a snapshot
operation initiated from user level through a ZFS ioctl. And the final line tells us
that this snapshot creation was done by the
(Much of this is from Matthew Ahrens of Delphix in the ZFS
developers mailing list,
and his message is (indirectly) how I found out about the
If this was a snapshot creation or deletion that had been done
rmdir, there would only be the
log entries (because obviously they use neither user-level ioctls
There seem to be any number of interesting internally logged ZFS events, but at this point I haven't really gone looking into this in any depth. I encourage people to look at this themselves for their own pools.