The idea of hierarchical filesystems doesn't feel like an API to me

May 15, 2022

One of the things that people say sometimes (including about serving static files on the web) is that "the filesystem is an API". On a superficial level this sounds fine. The abstracted form of a filesystem hierarchy is a common interface between programs, one that you can write code to use; you can create files and directories, inspect the hierarchy, read data that has been put in there, and delete things again (if you have permissions).

However, I've come to feel that the filesystem as a thing (abstract or otherwise) doesn't particularly feel like other APIs. Filesystems are program-independent hierarchical namespaces with some operations and types of entities that we all agree on (and a number of both that we don't; consider Unix symbolic links and hard links). But they aren't hard, nailed down interfaces for programs to code against in the way that other APIs are. The abstract idea of "the filesystem" has no defined content or structure (that's up to any particular use you make of the idea), and as far as actual program code goes we merely have a general agreement on the names for common operations and common filesystem contents ('files' and 'directories'). Different programming environments and even operating systems implement somewhat different actual APIs for filesystem access and manipulation, especially once you get out of the very basic operations.

(Even the idea of 'the contents of a file' is somewhat fuzzy. Is that a binary file or a text file? In some environments, the difference matters.)

If you say that the filesystem is an API, I feel that you're saying about as much as if you said that the web (as an idea and a general thing) is an API (which is true but in a very broad, architecture astronaut way). The filesystem being an API is pretty much the idea that you can use a commonly agreed on hierarchical namespace to communicate between programs, and between people and programs.

One place where this matters is if people aspire for other, more concrete APIs to become as broadly adopted and available as the filesystem "API". I feel that that would require people transforming those more concrete APIs into something more or less as broad, encompassing, and general as the idea of filesystems. For many things that people want to be broad APIs, I tend to not see any obvious path for that to happen because there doesn't seem to be a level between a broad idea and a concrete API.

(In part, there's a bit of me that bristles at calling the filesystem or the idea of filesystems an API, and I want to figure out what my objection is. I'm not sure I fully understand my objections yet.)

PS: You can use a filesystem to make a concrete API, or as part of a concrete API, but that's a different thing. You have one or more programs that's ascribing particular meaning to a hierarchy with specific contents, and specific manipulation of the contents may have specific effects.

Comments on this page:

The POSIX API is definitely a thing, and there are all sorts of discussions about the benefits of relaxing some of these requirements or guarantees in the context of NFS or HDFS. Things like locking, atomicité of operations and so on.

By Michael at 2022-05-16 03:42:15:

I don't think it's appropriate to call the file system an application programming interface.

It provides, but I don't think it's even appropriate to call it, a namespace.

A file system provides a mapping from a (hopefully) user-selectable, meaningful name to an essentially arbitrarily-sized blob of data and some metadata (the amount of which varies with different file system types), and typically but not necessarily also the ability to manipulate those mappings in various ways (read-only file systems exist; consider ISO 9660 or SquashFS). Really, that's about it. Yes, there's a gazillion more underlying details to how that is done, but at an abstract level, that's the utility of a file system.

That some application might put special meaning into the existence of, or the contents of, some particular files allows a file system to be used to enable (a form of) inter-process communication, but the actual API is then defined by that application; the file system simply provides a shared surface between the applications on either side of the communications channel. If you swap out one application for a different one that accomplishes the same task, it's unlikely that the two would respond identically (in an IPC fashion) to same-named files having the same contents. For example, I doubt that a typically configured AIX will respond as Linux does to a read or write against some file under /proc or /sys. Another example of IPC that does not involve a file system would be a HTTP request and response; in a broad sense that's still a form of IPC, though in that case, typically performed over a network connection.

Re: "there's a bit of me that bristles at calling the filesystem or the idea of filesystems an API, and I want to figure out what my objection is.":

Maybe a filesystem is more like a key-value store, where the keys are hierarchical. The only operations you can do on the store are CRUD.

So, it is/has an API, but not a very rich one ?

By Sam Birch at 2022-05-16 13:26:28:

"The filesystem is an API" has grated on me as well. My sense is that it's more useful to say that the filesystem is a calling convention that one can build APIs upon. Or perhaps one might call the filesystem a "protocol".

UNIX block and character device nodes (with all their ioctls) seem like an API to me. I'd say "/sys/block/* is an API" before I'd say "the filesystem is an API" or even "sysfs is an API". I think /etc/systemd probably counts as a configuration API for systemd and its various helpers, although it's not a wrapper around kernel function calls the way my first two examples are.

Written on 15 May 2022.
« The web is, in a sense, designed for serving static files
Python programs as wrappers versus filters of other Unix programs »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun May 15 20:55:51 2022
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.