From CPython bytecode up to function objects (in brief)

November 25, 2013

Python bytecode is the low level heart of (C)Python; it's what the CPython interpreter actually processes in order to run your Python code. The dis module is the heart of information on examining bytecode and on the bytecodes themselves. But CPython doesn't just run bytecode in isolation. In practice bytecode is always part of some other object, partly because bytecode by itself is not self-contained; it relies on various other things for context.

Bytecode by itself looks like this:

>>> fred.func_code.co_code

(That's authentic bytecode; you can feed it to dis.dis() to see what it means in isolation.)

I believe that Python bytecode is always found embedded in a code object. Code objects have two sorts of additional attributes; attributes which provide the necessary surrounding context that the bytecode itself needs, and attributes that just have information about the code that's useful for debugging. Examples of context attributes are co_consts, a tuple of constants used in the bytecode, and co_nlocals, the number of local variables that the code uses. Examples of information attributes are co_filename, co_firstlineno, and even co_varnames (which tells you what local variable N is called). Note that the context attributes are absolutely essential; bytecode is not self-contained and cannot be run in isolation without them. Many bytecodes simply do things like, say 'load constant 0'; if you don't know what constant 0 is, you're not going to get far with the bytecode. It is the code object that tells you this necessary stuff.

Most code objects are embedded in function objects (as the func_code attribute). Function objects supply some additional context attributes that are specific to using a piece of code as a function, as well as another collection of information about the function (most prominently func_doc, the function's docstring if any). As it happens, all of the special function attributes are documented reasonably well in the official Python data model, along with code objects and much more.

(Because I just looked it up, the mysterious func_dict property is another name for a function's __dict__ attribute, which is used to allow you to add arbitrary properties to a function. See PEP 232. Note that functions don't actually have a dictionary object attached to func_dict until you look at it or otherwise need it.)

Function objects themselves are frequently found embedded in instance method objects, which are used for methods on classes (whether bound to an object that's an instance of the class or unbound). But that's as far up the stack as I want to go today and anyways, instance method objects only have three attributes and they're all pretty obvious.

(If you have a class A with a method function fred, A.fred is actually an (unbound) instance method object. The fred function itself is A.fred.im_func, or if you want, A.__dict__["fred"].)

Note that not all code objects are embedded in function objects. For example, if you call compile() what you get back is a bare code object. I suspect that module level code winds up as a code object before getting run by the interpreter, but I haven't looked at the interpreter source to see so don't quote me on that.

(This entry was inspired by reading this introduction to the CPython interpreter (via Hacker News), which goes at things from the other direction.)

Written on 25 November 2013.
« Track your disk failures
The difference between CPython and Python »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Nov 25 23:11:13 2013
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.