Is concurrency 'hard'?

January 5, 2006

Is concurrency hard? You can certainly find people arguing both sides of the issue if you look hard enough (mostly for rather than against). But I think the question may be blurring two things together that are worthwhile to separate: 'concurrency is difficult to learn and understand', and 'concurrency is tricky to implement correctly'.

For example, in his reaction to Joel Spolsky's Java Schools article, Tim Bray gives a little taxonomy of difficulty:

My experience differs from Joel's in another respect: Recursion is mildly hard. Closures and continuations are hard. Concurrency is very hard. I never found pointers hard at all.

For me, it really depends on what sense of 'hard' one means. Closures and continuations are somewhat mind-bending to understand, but I've found closures easy to use in programs. In a way closures are so easy to use that you may not realize that you are using them, because things 'just work'. (I haven't really used a language with full continuation support, but Python's limited version is pretty easy to use too.)

(Arguably closures (and Python's limited continuations) are only hard to understand if you insist on knowing how the magic works.)

So is concurrency hard? My belief is that concurrency isn't particularly difficult to understand, but it is definitely difficult to use. I think we have lots and lots of experimental evidence of the latter, and implicit experimental evidence of the former in that there are lots of people who think they understand concurrency well enough to use it.

It's possible that concurrency is harder to understand than I remember. I suspect that knowing something about computer architecture and assembly language helped me when I was learning about it, so I don't know how it would hit people without that sort of a background. (Although I think any graduate of a good CS undergrad program ought to have been exposed to both.)


Comments on this page:

From 70.231.160.115 at 2006-01-05 20:07:43:

I think in context (launching off from a discussion about interviewing candidates), it's clear about "learning it well enough to be able to use it". If using it is hard, then in some sense "learning to use it well" is hard, and in some sense "learning about how to use it well" is hard.

It's sort of like chess. Your distinction here is roughly you can learn how chess is played, yet not be able to do it well. My distinction is that you can learn the rules for how the pieces move, without really learning how chess works. There's nothing in the rules about forks, pins, or even something simple like support. These are emergent concepts that come about when following the basic rules using any strategy that is at all approaching a winning strategy. I don't want to overstate the analogy, but I think concurrency is like that. Learning how a semaphor or a critical section or message rendezvous works is easy, learning how to use them to solve problems without bugs is hard--but it's only the latter that Joel and Tim Bray actually care about, because they're all that matters for actually programming, as opposed to pure academic exercises.

-- nothings

By DanielMartin at 2006-01-05 21:34:21:

I have heard from people more knowledgable in the ways of concurrent programming than myself that part of the reason concurrent programming is difficult, in terms of having so many hidden traps waiting for the unwary, is that we insist on building the wrong concurrency primitives into our languages.

Threads aren't the only possible way to structure concurrency - for example, http://www2.info.ucl.ac.be/people/PVR/bookcc.html

By cks at 2006-01-06 02:52:16:

I agree on threads; doing some justice to the whole subject is going to take me at least a whole entry, which may not come for a while.

From 70.231.160.115 at 2006-01-06 05:58:46:

One of the current pressures for concurrent programming is that CPU designers are running into a wall for getting further speedup for single threads. Thus hyperthreading and multiple cores on a single CPU offer more "total performance".

However, if the user primarily uses a single application at a time (as is common for the applications I develop--videogames), the only way to leverage that extra performance is through concurrent programming.

And here's the rub: the two most plausible ways of achieving bug-free concurrent programming (message passing a la erlang, and pure functional programming a la Haskell) both have significant performance losses, so that the win from multiple CPUs may well not make up for the lost performance due to the programming model.

Thus, right now, multiple CPUs as a solution to making programs faster is just a total boondoggle for programmers.

(As evidence of my comment regarding those programming models: For erlang, check the erlang FAQ about problems Erlang is not suited for. For pure functional programming, consider that the task of updating a hash-table-like object is O(1) in an imperative language with arrays, and is O(log N) for pure functional (you have to use binary trees suited to the task). As well, pure functional doesn't guarantee concurrency; if you need to update one of these structures repeatedly, that is, update the one returned by the previous call, you can't launch multiple of them in parallel at all. Simulations often have this sort of update structure, and videogames are essentially simulations.)

-- nothings

Written on 05 January 2006.
« Python synergies in list addressing
The old nameserver glue record hell »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Jan 5 02:33:36 2006
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.