ZFS's potentially very useful 'zpool history -i' option

January 11, 2017

I recently wrote a little thing praising zpool history. At the time I wrote that, I hadn't really read the manpage carefully enough to have noticed an important additional feature, which is zpool history's -i argument (and -l as well, sometimes). To quote the manpage, -i 'displays internally logged ZFS events in addition to user initiated events'. What this means in plain language is that 'zpool history -i' shows you a lot of what happened to your pool no matter how it was done. This may sound irrelevant and abstract, so let me give you a concrete example.

Did you know that you can create and delete snapshots in a filesystem by using mkdir and rmdir in the <filesystem>/.zfs/snapshot directory? If you have sufficient privileges (root is normally required), this works both locally and over NFS to a ZFS fileserver. Snapshots created and deleted this way don't show up in plain 'zpool history' because of course they weren't created with a 'zfs' command, but they do show up in 'zpool history -i'.

When you're looking at the output at this level, you will typically see three log events for a typical command:

<time> [txg:12227245] snapshot fs0-core-01/cs/mail@2017_01_10 (4350)
<time> ioctl snapshot

<time> zfs snapshot fs0-core-01/cs/mail@2017_01_10

The [txg:NNN] first line is the low-level internal log and is apparently the only log entry that's guaranteed to be there, I assume because it's written as part of the transaction; the remaining records can be lost if the machine fails at the right time or the program crashes, and they're written after the TXG record (as we see here). The ioctl entry tells us that this was a snapshot operation initiated from user level through a ZFS ioctl. And the final line tells us that this snapshot creation was done by the zfs command.

(Much of this is from Matthew Ahrens of Delphix in the ZFS developers mailing list, and his message is (indirectly) how I found out about the -i flag.)

If this was a snapshot creation or deletion that had been done through mkdir and rmdir, there would only be the [txg:NNN] log entries (because obviously they use neither user-level ioctls nor the zfs command).

There seem to be any number of interesting internally logged ZFS events, but at this point I haven't really gone looking into this in any depth. I encourage people to look at this themselves for their own pools.

Comments on this page:

By Aneurin Price at 2017-01-12 12:57:31:

I don't suppose you happen to know if there's a way to increase the size of the history buffer? I was looking for this the other day and concluded that the answer is almost certainly 'no', but hope springs eternal.

Right now 'zpool history' isn't even marginally useful to me, because I'm the only person using the pool and if it was recent enough to be in the history, then it's recent enough that I'll remember it (plus it's probably in bash_history anyway). This is a shame, because there have been a number of times that I've wanted to use it, but for it to be genuinely useful the size would need to be increased by a factor of maybe a hundred.

By cks at 2017-01-12 13:17:13:

Based on the code it appears not. History log size is apparently fixed at 0.1% of the pool size up to 1 GB (and a minimum size of 128 KB), according to the comments in spa_history.c in spa_history_create_obj.

(Awkwardly, I have to admit that up until you mentioned it I sort of vaguely thought that the log was endless, but that's clearly absurd. It just looks endless on an initial scan because the pool creation is never overwritten.)

That the size is just in the code does suggest that you could change it if you wanted to. However, things get tricky here; it appears that the current code never touches the history log size once it's created, which means that it won't change if the pool is later grown. That is at least annoying. (The size is persisted on disk if I'm reading the code correctly.)

Written on 11 January 2017.
« Picking FreeType CJK fonts for xterm on a modern Linux system
Modern ad networks are why adblockers are so effective »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Jan 11 01:47:29 2017
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.