== Python scopes and the CPython bytecode opcodes that they use Since I just got very curious about this, I want to write it down. CPython has four different families of bytecode opcodes for accessing variables, which are used under different circumstances and look in different places. They are: * ``*_FAST'' opcodes are used to access function local variables, which have [[a special storage scheme WhyLocalVarsAreFast]]. They're used in function scope for variables that are not part of a closure (including inside a closure for variables that are local to the closure alone). * ``*_DEREF'' opcodes are used to access [['cell variables' CPythonCellsClosures]] that are used as part of the implementation of closures; these are effectively function local variables with an extra level of indirection. They're used in function and closure scope for variables that are part of a closure. * ``*_GLOBAL'' opcodes are used to access variables that are explicitly known to be global variables or builtins, for example if you are in a function and have used _global_ on a variable name. * ``*_NAME'' opcodes are used to access variables in code at the module level or in a class that's being created, and are also the opcodes that _compile()_ generates. Effectively they are CPython's generic 'access a variable somehow' opcodes. (Update: under some relatively rare circumstances, NAME opcodes can also be used in functions. When and why is complicated enough to call for [[a full entry ExecScopingWeirdnessExplained]].) The first three opcode families look in the obvious places you'd expect. The NAME opcodes are a bit odd and to explain them I need to talk about stack frames briefly. CPython code always runs in the context of a stack frame. Ignoring implementation details for a moment, stack frames have three namespaces associated with them; the [[frame's locals MisleadingFLocals]], [[the globals FramesTrivia]], and [[the builtins SpelunkingFBuiltins]]. How the NAME opcodes work is that the ((LOAD_NAME)) bytecode looks at each of these namespaces in succession and takes the first instance of the name it finds, while ((STORE_NAME)) always writes things into the frame's locals. As the comments in the CPython source code put it: > [...] the name is treated as global until it is assigned to; then it > is treated as a local. (Technically something like this can also happen with ((LOAD_GLOBAL)) and ((STORE_GLOBAL)), since ((STORE_GLOBAL)) will not write to the builtins but ((LOAD_GLOBAL)) will look in them as a fallback.) When code is running at the module level, the frame's local namespace is the same as the global (module) namespace, but [[you can still copy from the builtins namespace to the module namespace FramesTrivia]]. When code is running during class creation, the local namespace is the class to be's namespace and the global namespace is the module namespace: > a = 10 > print globals() is locals() > class A(object): > a = a > print globals() is locals() This prints _True_ then _False_. (This adds yet another place in CPython that I know of where the left side of an assignment can be in a different namespace than the right side, updating [[my previous update FramesTrivia]]. One day I will have found them all.) PS: I suspect that this implementation of the NAME opcodes explains something I noticed a while ago, which is that [[the C level frame structure always has a real ((f_locals)) dictionary FramesTrivia]]. The code of the NAME opcodes directly uses this ((f_locals)) field, so having it always valid probably simplifies the innards of the bytecode interpreter. In fact if ((f_locals)) is NULL, the NAME opcodes throw an exception. (I'm not certain if it's ever possible to have a NULL ((f_locals)) under normal circumstances, although if you do a web search for 'no locals found when storing', 'no locals when loading', or 'no locals when deleting' you will get some hits.) === Sidebar: A bonus trivia contest with _compile()_ again If you've managed to follow my writing above, you should now be able to understand and explain what happens in the following code (a variant on [[yesterday's weirdness ExecScopeHandlingBug]]): > def exec_and_return(codeobj, x): > exec codeobj > return x > > a = 5 > co = compile("a = a + x; x = a", '', 'exec') > > print exec_and_return(co, 1), a This prints '_6 5_'. (This behavior is probably not strictly speaking a bug and it would be hard to fix given [[the constraints on _compile()_ ExecScopeHandlingBug]].) (Update: the explanation is now [[here ExecScopingWeirdnessExplained]].)