Wandering Thoughts archives

Using the tempfile module to get temporary files

Courtesy of Planet Python, I stumbled over Peter Bengtsson's puzzlement with Python's tempfile module, especially the mkstemp() function. It's an understandable puzzlement, because what's going on is fairly tied to Unix semantics (although Python supports mkstemp() on all platforms).

The tempfile module's mkstemp() is a relatively thin shim over the Unix mkstemp(3) library routine, which exists to securely create temporary files. This is more difficult than it looks; there are all sorts of race conditions, which can lead to security problems.

Update: I'm completely wrong here. Python's mkstemp() doesn't call mkstemp(3); instead it (re)implements the general algorithm and interface. This means that mkstemp() doesn't automatically use any special or necessary tricks done in your platform C library mkstemp(3).

Many of these issues are ultimately caused by the difference between file names and files. Put crudely, people exploit the race conditions by changing what file a given file name points to (there are a number of ways to do this). mkstemp(3) takes steps to guarantee that the file it creates is a new temporary file, but it cannot be absolutely sure that the file name will continue to point to it, so it returns the file (as well as the name).

Because this is Unix, the file is returned in the form of a file descriptor, which the os module deals with. The best way to do anything with this is to turn it into a Python file object with os.fdopen(). Sample code is:

import os, tempfile
def filetemp():
  (fd, fname) = tempfile.mkstemp()
  return (os.fdopen(fd, "w+b"), fname)

This is more or less what the tempfile module's TemporaryFile class does to return a file-like object. (They are real file objects on Unix but wrapped objects on other platforms.)

(The OpenBSD mkstemp(3) manpage has some additional information and commentary that may be useful.)