Wandering Thoughts archives

ZFS: Reservations versus quotas

Suppose that you have a a number of groups (or people), each of which is purchased a certain amount of disk space, and that you have put all of the groups in the same overall ZFS pool. ZFS gives you two ways to limit each group's disk usage to the amount of space that they're entitled to: reservations or quotas. Which one is better?

It turns out that the answer is simple: use quotas.

(In both cases you give each group a top level container filesystem and give it a quota or a reservation of however much they have. With reservations you need another 'system' container that reserves any remaining space.)

Quotas have at least two advantages over reservations for this:

• df gives more useful output. With quotas, the 'size' of a filesystem is the group's quota; with reservations, the size is the total pool size (although the 'available' figure is correct).

• snapshots work better if the group has filled up their space. With reservations, creating a snapshot fails (with an out of space error); with quotas, users just can't remove or modify files (they get a quota exceeded error) until you remove the snapshot.

(This is still inconvenient, but better overall than no snapshot at all.)

Since you can't create snapshots if your entire pool fills up, you'll still want to make sure that your pool has a little bit of spare space that no group can use. This is easy to do with quotas but not so much with reservations.

Nested quotas work right (or at least the way I want them to work); you cannot accidentally override a higher-level quota by giving a sub-filesystem a big quota value.