Wandering Thoughts archives

Multilevel list comprehensions in Python

Python has recently (at least for some values of recently) grown 'list comprehensions', which let you easily iterate over a list to transform or select entries (or both). List comprehensions can be thought of as syntactic sugar for map and filter operations, but they're actually more powerful.

One reason is that you can write a multilevel list comprehension, which effectively iterates over multiple levels of lists. Take the case where you have a list within a list and want to return all of the low-level elements as a list:

l = []
for rr in qa:
  for s in rr.strings:
    l.append(s)

This can be rewritten as a two-level list comprehension:

l = [s for rr in qa for s in rr.strings]

This can't easily be done via map. (We would probably have to roll in a reduce to flatten the list of lists that map would give us into a single-level list.)

Multilevel list comprehensions work left to right; the leftmost 'for X in Y' is the outermost one, and then we step inwards as we move right. You can also use if conditions, so the correct version of the list comprehension I wrote, in context and with error checking, would be:

from dns.rdatatype import TXT
l = [s for rr in qa if rr.rdtype == TXT \
     for s in rr.strings]

What impresses me about Python is that this works just the way I thought it would work and both of these examples worked the first time, just as I wrote them, and needed no debugging. (The first version actually got used in a scratch program.)