Wandering Thoughts archives

Nested conditional expressions in Python (and code golf)

Recently I had an occasion to use a nested (or chained) conditional expression. I haven't used conditional expressions much, so at first I just wrote out what struck me as the obvious way:

res = a.field1 if a.field1 else obj.field2 if obj else None

(The goal is to use a.field1 if it's got a value, obj.field2 if obj is there, and otherwise None.)

Then I paused to ask myself if this was going to have the right grouping of evaluation; testing said that it did, to my pleasant surprise. It's always nice when Python behaves the way I expected it to and my naive code works. That it happens so often is one of the reasons that I like Python so much.

While this nested conditional expression was the obvious way to write the expression (since I was converting it from what would otherwise be a nested if), it's possible to be more clever. The simple way is to get rid of the actual conditional expressions in favour of exploiting the side effects of or and and:

res = a.field1 or (obj and obj.field2) or None

(Given when I'm trying to do here this doesn't suffer from the usual problem of (ab)using and and or this way.)

Of course we can golf this code further:

res = a.field1 or getattr(obj, 'field2', None)

To my mind this is well over the line into excessively clever, partly because it mixes two different ways to refer to fields in the same expression. Even the first condensed version is not something I'm entirely happy with (partly because it's subtly different than the straightforward version using conditional expressions). So my initial version is going to stay in my code.

(I think I've basically recanted on my views about avoiding conditional expressions in Python by now. Time moves on and I get used to things.)

The pain (or annoyance) of deploying a simple WSGI thing

It started on Twitter:

@eevee: it is time once again to set up a small innocuous wsgi thing and i am again reminded that this sucks and i want to fix it so bad

@thatcks: If only deploying a WSGI thing was as easy as PHP. And I wish I was making a joke there.

(Note that @eevee is the author of the infamous rant PHP: a fractal of bad design.)

Some number of people are now thinking that there's no problem here and that WSGI apps are pretty easy to deploy. After all there's lots of WSGI servers, many of them quite good, and it's not like it's hard to hook one up to your main (or frontend) web server. An entry here, an entry there, a new daemon process, and you're done. Maybe you even use Apache's mod_wsgi, which gets it down to a configuration entry (and a server restart, but you probably needed that anyways).

Well, here's the simple PHP deployment process: put a .php file in the appropriate spot in your web server's document root. You're done.

(As noted by @bobpoekert, CGIs also have basically this property.)

Yes, yes, of course there is a great pile of stuff behind the scenes to make that work. And of course it isn't as flexible and as scalable as the full bore WSGI version. But it demonstrates what a simple deployment actually is and frankly a simple basic deployment is all that 99% of all web apps need (the existence proof is all of the basic PHP apps). Even a relatively full-featured WSGI deployment should only require two files and nothing else (one actual .wsgi file and one file to describe things like what URL it connects to), with the pieces to make it work integrated with your web server.

(The actual running of your WSGI app could be in a separate container daemon that's handled by a separate supervision process. That's an implementation detail that you shouldn't have to care about for a simple WSGI deploy process any more than you should have to care about the details of how your web server communicates with your WSGI app.)

As a side note, as a sysadmin I happen to think that standalone daemons are both a pain in the rear and utterly the wrong approach for a scalable deployment of lots of apps with unpredictable load. But that's another blog entry.

Why I think dir() excludes metaclass methods

I was recently reading David Halter's Why Python's dir function is wrong which points out that dir() on classes and types excludes some valid attributes from the result (for example, __bases__). As it happens, I have a theory about why Python behaves this way. The short version is that it is a heuristic to make dir() more useful.

(Note that classes and types are the same thing. From now on I'm going to use 'class' to mean both.)

When you use dir() on a class there are at least two things you can be interested in, namely what attributes will be visible on an instance of that class (ie, what attributes are defined by the class and its inheritance hierarchy) and what attributes are visible on the class itself (more or less). My feeling is that almost all of the time people use dir() they are more interested in the former question; in fact, I'd expect that many people don't even know that you can have attributes visible on a class but not its instances.

Even ignoring the direct usability issue, doing dir() 'properly' has a couple of other significant drawbacks. First, you lose any good way to find out what attributes will be visible on instances of classes; you'll wind up wanting to add a special flag to dir() to return to the old behavior. Second, the result of dir(anyclass) is likely to be confusing in practice because it will mingle instance-visible and class-only attributes, including reasonably common special methods. Most obviously, every class has a __call__ special method from type() but it can only be called on the class itself.

It's probably worth mentioning the obvious thing, namely that metaclasses didn't used to exist while dir() is very old. Effectively this set the behavior of dir() as excluding metaclasses; you can imagine the uproar if dir() had suddenly added a bunch of methods when used on classes (including giving __call__ to everyone) when metaclasses were introduced. This might well have been (correctly) regarded as a significant change from existing behavior.

(This seems especially likely as I believe that there is some code that uses dir() on classes for introspection.)

Also I've played fast and loose with something here, because dir() is actually not the list of what attributes will be visible on anything. dir() is the list of attributes that are defined on particular things, but additional attributes can be materialized in various ways and defined attributes can even be made inaccessible if you try hard enough. This may suggest why any change in dir()'s behavior has not been a high priority for Python developers; in a sense you usually shouldn't be using it in the first place. And (as pointed out on reddit) this dir() behavior is explicitly documented and it does even more magic than this.

PS: for more on metaclasses and many low level details about them, see my index to my metaclass series.

Sidebar: what additional attributes you would see on normal classes

You can get this list in a simple way:

sorted(set(dir(type)) - set(dir(object)))

Rather than put a big chunk of output here I think I'm going to leave it to my readers to run this for themselves if they're interested.

Classes with custom metaclasses would add any additional attributes from those custom metaclasses.