Wandering Thoughts: Recent Entries For 2005/11/28

How not to set up your DNS (part 3)

Presented once again in illustration:

; sdig ns isotech.com.cy
dns2.lordosinfo.com.
dns1.lordosinfo.com.
; sdig ns lordosinfo.com.
dns1.edynet.com.cy.
dns1.lordosinfo.com.
; sdig ns edynet.com.cy.
dns2.netconnect.com.cy.
dns1.netconnect.com.cy.

At the moment:

• dns2.netconnect.com.cy is not responding,
• dns1.edynet.com.cy doesn't exist according to dns1.netconnect.com.cy,
• dns2.lordosinfo.com is a CNAME to outopos.lordosinfo.com,
• and outopos.lordosinfo.com is a CNAME to itself (that's good for bonus points).

In the face of all of this, our DNS server is currently laughing hysterically at the very idea of looking up A and MX records for isotech.com.cy. (In fact it bitches about 'protocol error', possibly due to the nice CNAME loop.)

What Python's global interpreter lock does (and doesn't) protect

Most people doing threading in Python know about Python's Global Interpreter Lock (GIL), which causes only one thread to be running in the CPython interpreter at any one time. This means that threaded code already has a certain amount of implicit locking going on, making certain operations thread-atomic without you writing explicit locks.

The important thing about the GIL for this is that it protects bytecodes, not Python statements. If something happens in a single bytecode, it's protected; otherwise, you need explicit locking. The most important thing done in single bytecodes is calls to methods implemented in C, such as all of the methods on builtin types. So things like list .append() or .pop() are atomic.

(All bets are off if you're dealing with someone's Python subclass of a builtin type, since it depends on what exactly they overrode.)

An important note is that in-place operations like '+=' are not actually single bytecodes, so counters like 'self.active += 1' need locks.

(Because they must be able to work on immutable objects, the actual INPLACE_* bytecodes result in an object, which is then stored back to the left hand side in a separate instruction. In-place operations on mutable builtin types can be atomic, but there aren't many mutable builtin types that support '+' et al.)

By default, Python only switches between runnable threads every hundred bytecodes or so, which can disguise code that isn't threadsafe. You can make Python switch almost every bytecode with sys.setcheckinterval(); see the sys module documentation. (Python opcodes don't always allow thread switching after themselves, so it's never literally every bytecode.)

(Again we see that implementation details matter, although there is an argument that this is too much black magic.)