Modern disk write caches and how they get dealt with (a quick overview)

October 25, 2013

Basically all modern disks (SAS, SATA, etc, it doesn't matter) have write caches. I think that most disks these days default to having them on, even in the 'enterprise' space, since enterprise OSes have generally been dealing with disk write caches for some time and so are safe in the face of them.

(In the old days when write caches were just starting to appear, the stereotype was that consumer drives defaulted to enabling it and enterprise drives to disabling it.)

Generally, if you have a disk with its write cache enabled ('WCE' in the jargon) there are three things you can do to force writes to be committed. The blunt hammer is disabling the write cache ('WCD'), which I believe generally has such a bad effect on performance that you don't want to do that and in fact any number of OS and filesystem setups will turn the write cache on for you if it's turned off (cf).

I believe that all modern disks with write caches support a way to flush the cache; in SCSI (and SAS) this is the SYNCHRONIZE CACHE SCSI command. Of course this has the usual drawback that it flushes even data that you may not care about and not need to be committed to disk right now. As I discovered yesterday, drives can also support a write option called 'Force Unit Access' (FUA) that bypasses the write cache in order to force what you're writing to be forced to disk. In general FUA is bundled with another feature called 'Disable Page Out' (DPO), which tells the drive that putting the data into cache is not useful.

(DPO and FUA can also be used on reads, apparently, but I haven't looked into that at all.)

Historically, FUA support has apparently been common and well done in SCSI (and SAS) drives and SAS drivers routinely support it. FUA support in ordinary SATA drives was apparently a chancy thing in the beginning and as a result Linux's SATA driver still defaults to pretending that drives do not support FUA (and I believe DPO) regardless of what the actual drive reports (other OSes may behave similarly). How SATA drives behind a SAS controller behave and how your SAS controller driver treats them is probably an interesting question that you may want to check for yourself if you have such a setup.

Of course higher level software may make some or all of this academic. Your filesystem may also only bother with write cache flushes instead of trying to be more fancy with selective FUA usage.

(This entry is due to me getting both curious and nervous after discovering FUA in yesterday's entry and then discovering that drives in our SATA to SAS environment claimed to support it when no directly connected SATA drives did.)

Written on 25 October 2013.
« How to force a disk write cache flush operation on Linux
10G Ethernet and network buffer sizes (at least on Linux) »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Oct 25 00:57:02 2013
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.