A lingering sign of old hopes for ZFS deduplication
Over on Twitter, I said:
It's funny-sad that ZFS dedup was considered such an important feature when it launched that 'zpool list' had a DEDUP field added, even for systems with no dedup ever enabled. Maybe someday zpool status will drop that field in the default output.
For people who have never seen it, here is '
zpool list' output
on a current (development) version of OpenZFS
; zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT ssddata 596G 272G 324G - - 40% 45% 1.00x ONLINE -
DEDUP field is the ratio of space saved by deduplication,
expressed as a multiplier (from the allocated space after deduplication
to what it would be without deduplication). It's always present in
zpool list' output, and since almost all ZFS pools don't use
deduplication, it's almost always 1.00x.
It seems very likely that Sun and the Solaris ZFS developers had
great hope for ZFS deduplication when the feature was initially
launched. Certainly the feature was very attention getting and
superficially attractive; back a decade ago, people had heard of
it and would recommend it casually, although
actual Solaris developers were more nuanced.
It seems very likely that the presence of a
DEDUP field in the
zpool list' output is a product of an assumption that
ZFS deduplication would be commonly used and so showing the field
was both useful and important.
However, things did not turn out that way. ZFS deduplication is almost never used, because once people tried to use it for real they discovered that it was mostly toxic, primarily because of high memory requirements. Yet the DEDUP field lingers on in the default 'zpool list' output, and people like me can see it as a funny and sad reminder of the initial hopes for ZFS deduplication.
(OpenZFS could either remove it or, if possible, replace it with the overall compression ratio multiplier for the pool, since many pools these days turn on compression. You would still want to have DEDUP available as a field in some version of 'zpool list' output, since the information doesn't seem to be readily available anywhere else.)
PS: Since I looked it up, ZFS deduplication was introduced in Oracle Solaris 11 for people using Solaris, which came out in November of 2011. It was available earlier for people using OpenSolaris, Illumos, and derivatives. Wikipedia says that it was added to OpenSolaris toward the end of 2009 and first appeared in OpenSolaris build 128, released in early December of 2009.