== How _zpool status_ reports on ZFS scrubs and resilvers

A recent thread on the ZFS mailing list concerned inaccurate or
misleading reports from _zpool status_ about the progress of scrubs
(both bad time estimates and a scrub that wasn't finishing despite
claiming to be 100% done). As it happens, I know something about how
this works because I recently went digging into what information the
kernel actually reports to userland.

The kernel reports the following information:

* the number of bytes currently allocated in the pool and in each vdev.

* how many bytes in the pool have been examined by the scrub or
  resilver; on current versions of Solaris, it also reports
  per-vdev bytes examined as well.

  (When resilvering the kernel also reports how many bytes have
  been repaired in the pool and on each vdev.)

* the time that the scrub or resilver has been running.

Because _zpool status_ only gets a snapshot of one moment in the scrub
it can only make a straightforward extrapolation of how much time it
will take to scrub the whole pool. So if the scrub rate initially starts
out quite fast but then slows down later due to fragmentation (because
[[ZFS does not scrub linearly ZFSNonlinearScrubs]]) or a bunch of user
IO interfering with the scrub, your time to completion will bounce
around significantly.

(Current versions of _zpool_ also report how long the scrub has been
running, which is in many ways a much more useful number.)

Now, observe something important: there is no explicit count
of how many bytes there are left to scrub. _zpool status_
simply assumes that 'allocated - scrubbed' is how many bytes
are remaining to scrub, but this isn't necessarily the case. In
some situations it's possible for ZFS to have scrubbed more
bytes than are actually allocated in the pool, resulting in
scrubs reaching 100% without finishing (per [[this bug report
http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6899970]]).

(This doesn't necessarily mean that ZFS scrubs chase updates; you could
equally well get into this situation by deleting a big snapshot after
ZFS had scrubbed it but before the scrub finishes. This will immediately
drop the bytes allocated count without affecting bytes scrubbed.)

PS: on inspecting current OpenSolaris source code, I see that it's
recently moved to a more complicated scheme where it tries to keep
better track of these numbers, likely as a consequence of the above
bug. This will presumably appear in Solaris in some future patch or
update.

PPS: contrary to what various sources will tell you (including the above
bug report), the Solaris kernel really does report these statistics in
*bytes*, not blocks.