Python 3 from the perspective of a Unix sysadmin

December 17, 2011

I've been thinking about Python 3 for a while, mulling over things like how I feel about it and how likely I am to use it, and I've decided that one reason my feelings are complex is that I have three different views of it, from three different perspectives. Today is the day for the first perspective: Python 3 from the perspective of a Unix sysadmin who uses Python to program important parts of our systems.

I don't have any way to put this nicely, so I'll say it right up front: for a Unix sysadmin, Python 3 is currently highly radioactive and should be completely avoided. Our current systems are written in Python 2; there is no prospect of this changing and I am going to keep writing sysadmin things in Python 2 for the indefinite future. I will stop this only when the systems we use stop packaging Python 2, and I certainly hope that that doesn't happen for, oh, a decade or more.

The fundamental problem is that Python 3 wants the operating system environment to be Unicode, and Unix is not. When Python 3 comes into contact with messy reality, bad things happen and things fail. These failures are vaguely tolerable for ordinary user programs; they are intolerable for programs used for system management. I cannot afford to write programs that silently omit names from os.listdir()'s results, that don't see some environment variables sometimes, or that die with mysterious error messages if given the wrong arguments. There are workarounds for some of these issues (but none yet for the sys.argv issue), but they are limited in scope and unlikely to be pervasive (in, eg, third party modules that I want to use).

So long as Python 3 is busy denying Unix reality (and causing all sorts of complications as a result of this), the sysadmin side of me can't and isn't going to touch it. I doubt that the Python 3 developers care about this and I doubt that anything is going to change in Python 3, which is kind of a pity.

(I could probably write system tools in Python 3 if I wanted to and tried hard enough and had to, but I don't see any reason to do so given that Python 2 is there and going to be there for a long time to come. Python 2 works, it works without huge contortions, and I don't really see anything compelling in Python 3 so far.)

Sidebar: on the long term availability of Python 2

At this point in time I see essentially no prospect of Python 2 being removed from Linux distributions in the next five years (minimum). The very first step along the long path of removing Python 2 would be for distributions to migrate Python based system tools from Python 2 to Python 3, and that hasn't even started yet (distributions are just now starting to talk about maybe moving some of their Python-based tools to Python 3 for their next release).

The chances of Python 2 disappearing any time soon from more conservative and slow moving Unixes like FreeBSD and Solaris (and Mac OS X) are best described as 'laughable'.

Comments on this page:

From at 2011-12-18 14:44:37:

I believe you're out of date on this. PEP 383 describes the solution, and it seems to work for sys.argv.

$ python -V
Python 2.7.1+
$ python
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> open('\xff', 'w')
<open file '�', mode 'w' at 0xb76d3e90>
$ python3 -V
Python 3.2
$ python3 -i ? ?
>>> import sys
>>> sys.argv
['\udcff', '\udcff']
>>> open(sys.argv[1], 'r')
<_io.TextIOWrapper name='\udcff' mode='r' encoding='UTF-8'>
>>> print (sys.argv[0])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcff' in position 0: surrogates not allowed
By cks at 2011-12-18 18:15:32:

That's pretty good news (oddly it's missing from the What's New writeups, so I can't tell which 3.x version it was introduced in). I could grumble that you'll get an exception if you try to print anything involving such arguments (eg, an error message), but it's a significant improvement on the previous situation.

From at 2011-12-19 20:17:29:

I agree that python3, will have to slow adoption for sysadmins. And most of us will probably start using it when it's on all of our systems. What about perl6, it's been on the horizon for ages?

By cks at 2011-12-19 22:15:35:

I haven't used Perl for years and I'm completely out of touch with the Perl world; anything I could say would be hearsay and gossip.

From at 2011-12-20 03:21:54:

Perl 6 is a different language than Perl 5 at this point. They are clearly related to each other, and some Perl 5 programs will even work in Perl 6 with no changes, but the languages are composed out of very different sets of axioms, so their respective idiomatic approaches to any at all non-trivial tasks differ. Perl 6 is still in development, but there have been implementations covering the spec to various degrees for years. The spec continues to evolve at the margins as people try to implement it in entirety and find ambiguities and difficulties, but those efforts are symbiotic. Wide swathes of the spec are stable and implemented in several different runtimes, several of which recently got fast enough that it feels foreseeable to use Perl 6 in small actual work soon (instead of just for dabbling). Lots of modules are as yet missing though so it’s not yet a very productive language to work in. It will be a while yet before sysadmins have much to do with it.

Meanwhile after a long time of suffering an ill release process that caused Perl 5 to evolve very slowly, it switched from feature- to time-based releases a few years ago and sledded off. It’s improving at a speedy pace now, evolving and improving existing features, sometimes adding new ones, sometimes stealing ideas from Perl 6 (for good and for bad) (though implementing them in its very different way (for good and for bad)). Its Unicode support is approaching the point of ideal, and is far more complete and correct than any other language’s at the time of writing, thanks to a recent push to iron out some remaining divergences between its string model and its implementation. There have been tons of infrastructural improvements in the CPAN tool chain in recent time. And a lot of very important stuff has gone on on the CPAN. There is a new object model called Moose which picks up a lot of concepts that were developed in the ’80s – most importantly polymorphism and code reuse by mechanisms other than inheritance. A big rush of development went into an HTTP adapter protocol and implementation (called PSGI and Plack respectively), originally repurposing code written for the Catalyst web framework into design ideas stolen back from Python’s WSGI and Ruby’s Rack. I think that just about covers the biggest and most important stuff… I could go on for quite a while about all the smaller cool things that have happened, there’s just so much going on.

That’s where the Perl world(s!) stand(s) these days, in brief.

Aristotle Pagaltzis

Written on 17 December 2011.
« Avoiding the classic C quoting bug in your language
Understanding the close() and EINTR situation »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Dec 17 02:59:31 2011
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.