Wandering Thoughts archives

dict.setdefault() as a concurrency primitive

One of the things I do in the back of my mind is to keep my eyes open for Python things that could be useful for concurrency (especially since I still have a concurrency problem that I have to solve one of these days). Because of what the GIL protects, one of the useful things to look for is interesting operations on builtin types.

All of which is a roundabout way of saying that it recently struck me that dict.setdefault() is a limited test-and-set operation. All the ingredients are there: setdefault() tests the 'value' of something, will replace it with your value if it matches the test, and returns the result. It is limited in that you can only test for a single value, 'unset'.

As an example, we can use this to write acquire and release as:

def acquire(d, k, me):
     r = d.setdefault(k, me)
     return r == me

def release(d, k):
     del d[k]

(This has one danger; acquire() will succeed when you already have acquired the resource, but release() does not guard against that. You could solve this by having acquire() make up a random identifier instead of using one that the caller comes up with.)

While it may seem perverse to come up with new concurrency primitives when Python already has some, these work even if you are not using threads (for example, if you are using signals inside a single process). And yes, it is a bit perverse, but I like the challenge of this sort of thing.

(The thread module doesn't say that its locks have thread-related side effects if you always invoke them as non-blocking, but it doesn't guarantee that they don't. And forgetting to make an acquire non-blocking will blow your foot off in a non-threaded program.)