What Python's global interpreter lock does (and doesn't) protect
Most people doing threading in Python know about Python's Global Interpreter Lock (GIL), which causes only one thread to be running in the CPython interpreter at any one time. This means that threaded code already has a certain amount of implicit locking going on, making certain operations thread-atomic without you writing explicit locks.
The important thing about the GIL for this is that it protects
bytecodes, not Python statements. If something happens in a single
bytecode, it's protected; otherwise, you need explicit locking. The
most important thing done in single bytecodes is calls to methods
implemented in C, such as all of the methods on builtin types. So
things like list .append()
or .pop()
are atomic.
(All bets are off if you're dealing with someone's Python subclass of a builtin type, since it depends on what exactly they overrode.)
An important note is that in-place operations like '+=
' are not
actually single bytecodes, so counters like 'self.active += 1
' need
locks.
(Because they must be able to work on immutable objects, the actual
INPLACE_*
bytecodes result in an object, which is then stored back
to the left hand side in a separate instruction. In-place operations
on mutable builtin types can be atomic, but there aren't many mutable
builtin types that support '+
' et al.)
By default, Python only switches between runnable threads every
hundred bytecodes or so, which can disguise code that isn't
threadsafe. You can make Python switch almost every bytecode
with sys.setcheckinterval()
; see the
sys module documentation.
(Python opcodes don't always allow thread switching after themselves,
so it's never literally every bytecode.)
(Again we see that implementation details matter, although there is an argument that this is too much black magic.)
|
|