Our Python fileserver management code has been quite durable over the years

July 3, 2023

At this point we've been running our ZFS based NFS fileserver environment for about fifteen years, starting with Solaris 10 and evolving over time to the current Ubuntu 22.04 based servers. Over that time we've managed the various iterations of the same basic thing primarily through a local set of programs (all with names starting in 'san', despite the fact that we don't have a SAN any more). These programs have always been written in Python. They started out as Python 2 programs on Solaris and then OmniOS, and were moved to Python 3 when we moved to our Linux based fileservers. Naturally, we have version control history for the Python code of these tools that goes all the way back to the first versions in 2008.

(For reasons, the Solaris and the Linux versions are in different source repositories.)

I was recently working on these programs, which made me curious to see how much the current ones have changed from the very first versions. The answer turns out to be not very much, and only in two real areas. The first is that in the change from Python 2 to Python 3, we stopped using pychecker annotations and the optparse module, switching to argparse (and making a few other Python 3 changes). The second is that when we moved from the OmniOS fileserver generation to the Linux fileserver generation, we moved from using iSCSI disks that came from iSCSI backends (and Solaris/OmniOS device names) to using locally attached disks with, naturally, Linux device names. Otherwise, the code is almost entirely the same. Well, for features that have always existed, since we added features to the tools over time. But even there, most of the features were present by the end of the OmniOS era, and their code mostly hasn't changed between then and now.

(In some programs, more comments changed than code did. This has left some vaguely amusing artifacts behind, like a set of local variables cryptically called 'bh', 'bd', and 'bl', which were originally short for 'backend host/disk/lun'. We no longer have hosts or LUNs, but we still have things that fill in those slots and I never renamed the local variables.)

On the one hand, this is what you'd want in a programming language; when and if there's change, it's because you're changing what the code does and how it does it, not because the language environment has changed or works differently on different systems. On the other hand, these days it feels like some programming environments exist in a constant state of churn, with old code either directly obsolete or functionally obsolete within a few years due to changes around it. Python hasn't been without such changes (see Python 2 to Python 3), but in practice a lot of code really has carried on basically as-is. This is something we rather appreciate in our local tools, because our real goal isn't to write and maintain tools, it's to do things with them.


Comments on this page:

I wrote a longer version of this, but this also points to the santools' code being "boring." Code that only uses functions and modules isn't using features that are the subject of academic research. The environmental changes (such as the rise of generators and async/await) do not perturb them. Even adding things like keyword-only arguments don't require changes to code that came before.

To put it simply, the code is only stable over time if it is built on stable foundations. It's just that programmers (non-sysadmin) love the new shiny things.

Written on 03 July 2023.
« The evolving Unix attitudes on handling signals in your code
Web servers should refuse requests for random, unnecessary URLs »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Jul 3 23:29:44 2023
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.