The problem with compiling your own version of Python 3

January 6, 2014

I've previously mentioned in passing that I simply can't use Python 3 on some platforms because it's not there (or perhaps it's there only in an old and lacking version). As reported by Twirrim, in some places the popular answer to this issue is to say that I should just compile my own local version of Python 3 by hand (perhaps in a virtualenv). At this point most of the sysadmins in the audience are starting to get out of their chairs, but hold on for a moment; I want to make a general argument.

There is a spectrum of Python coding that ranges from big core systems that are very important down to casual utilities. For something that is already big and complex, the extra overhead of compiling a specific version of Python is small (you've probably already got complex installation and management procedures even if you've automated them) and can potentially increase the reliability of the result. Nor is the extra disk space for another copy of the Python interpreter likely to be a problem; even if the disk space used by your system doesn't dwarf it, your core system is important enough that the disk space doesn't matter. But all of this turns on its head for sufficiently little and casual utilities; because they're so small, building and maintaining a custom Python interpreter drastically increases the amount of effort required for them as a total system.

Somewhere on the line between big core systems and little casual utilities is an inflection point where the pragmatic costs of a custom Python interpreter exceed either or both of the benefits from the utilities and the benefits from using Python 3 instead of whatever version of Python 2 you already have. Once you hit it, 'install Python 3 to use this' ceases being a viable approach. Where exactly this inflection point is varies based on local conditions (including how you feel about various issues involved), but I argue that it always exists. So there are always going to be situations where you can't use your own version of Python 3 because the costs exceed the benefits.

(With that settled, the sysadmins can now come out of their chairs to argue about exactly where the inflection point is and point out any number of issues that push it much further up the scale of systems than one might expect.)


Comments on this page:

By dozzie at 2014-01-07 08:45:01:

Add one more thing to the whole matter: the cost of maintaining one more language in the whole environment.

I use various languages, ranging from C, through shell, Perl, Python 2, Ruby, ending on my most recent love, Erlang. Every one of them has its strengths. What are benefits of C and shell are quite obvious. Perl has still enormous amount of modules in CPAN, and historically it's the most common language for tools in my environment. Python 2 comes with large subset of what I need, and adding the fact it's installed on Red Hat by default, it's good for bootstrapping configuration system. In Ruby people have written excellent data transporting daemons, like Fluentd and logstash, so my own plugins are written in Ruby. Erlang shines when I need to contact all the systems I manage.

All of the above-mentioned languages have their use and strengths. And now comes Python 3, which doesn't come with amazing concurrency model, so it can't compete with Erlang. It's not the language OS is written or operated in, so it won't replace C nor shell. It's not installed by default with OS, so it's not a good replacement for Python 2. Little libraries -- Perl won't go away. It's not used by daemons I rely on, so Ruby is to stay.

Out of that all, there's no killer-feature of using Python 3. It would be using the language just for the sake of using the language. I already have quite large toolbox and some of my colleagues don't keep up with it. While adding Python 3 to the set simply doesn't make my life easier, it would actively make their life harder.

Written on 06 January 2014.
« Some thoughts on blog front pages in the modern era
The good and bad of Linux's NetworkManager »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Jan 6 23:48:11 2014
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.