The periodic strangeness of idiomatic Python

July 29, 2012

Suppose that you want to do something N times, for whatever reason. In C, the straightforward and idiomatic way to do this is a for loop; 'for (i = 0; i < times; i++) { .... }'. Since Python doesn't have this form of a for loop, the Python equivalent is a while loop. However, many people would probably say that this isn't idiomatic Python. What I think of as the idiomatic Python way to do 'do something N times' is:

for _ in range(0, times):

(Some people will use xrange() instead of range() here.)

This is certainly what instantly popped into my head when I ran into this situation recently and at first I didn't think any more of it. But once I began actually looking at this it started getting stranger and stranger, less like a clear language idiom and much more like a convention. Let me run down a number of the ways that this is strange:

  • It's a rather indirect way of expressing 'do something N times'. The C for loop is pretty direct by contrast.

    (With that said, I'm not sure a while loop would be that much more direct. The directness advantage that C has is that all parts of the for loop's control are there in one chunk; a while loop spreads them out in three different lines.)

  • We're doing things in this odd way partly to use as many builtins as possible, often in the name of (nominal) efficiency. Yes, this avoids a couple of extra lines to initialize and increment an otherwise unused counter, but I don't think that really makes it clearer.
  • In the pursuit of this idiom we're creating a list or at least an iterator and walking it, throwing away the result. In many languages this would be wince-inducingly inefficient (or at least much worse than basic integer arithmetic with a variable). It's a (probable) win in CPython because of the whole builtins vs non-builtins issue.

    (Not only is range() a builtin, but for with iterators has direct bytecode support.)

  • You pretty much need to know this idiom in order to understand this code without a bunch of thought (which is not the case for the C version). A special tricky point is the use of `_' as a special variable name used to indicate 'I don't care about this variable, I just have to have something here'; this is entirely a convention in (some) Python programming circles, with no special meaning in the language itself.

    (As a corollary, I doubt that this is an idiom that would naturally occur to people who are not already immersed in Python.)

  • When using this idiom you'd better remember the exact effects of range()/xrange(), since eg 'range(1, times)' is very much not what you want.

    (Again the C equivalent has this clearly visible.)

The overall summary of this is that the Python idiom really is close to being an idiom, in the literal definition of the word: it is an expression whose meaning is not clearly and immediately understandable from a quick read of its component parts. By contrast the C idiom is much clearer (at least for me).

(I don't think that all of this makes the Python idiom bad; it remains the most compact and probably the most efficient way of expressing this. And even without knowing this idiom off the top of your head I think it's reasonably clear roughly what it does (and it's reasonably easy to work out all of the details).)

Written on 29 July 2012.
« The ecological niches of current open source Unixes
IPv6 is going to be a fruitful source of configuration mistakes »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Jul 29 01:21:56 2012
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.