Wandering Thoughts: Recent Entries For 2005/10/28

Inside building RPMs with Python distutils

For reasons outlined in CPANProblem, I don't install Python packages using their setup tools; I build them into RPMs and install the RPMs. Python's distutils (the setup system used by most Python packages) have a command to do this, called bdist_rpm, but it has some issues that have caused me to become quite familiar with its inner workings.

bdist_rpm works in the following steps:

1. use the distutils sdist command to create a source tarball.
2. write an RPM specfile.
3. create a local RPM build tree and put the specfile and tarball into it.
4. run rpmbuild in the local RPM build tree to create the source or binary RPM.

My experience is that the simpler the package and the package's setup.py, the more likely that everything will go fine. Otherwise, you can run into a variety of problems.

Because of step #1, bdist_rpm is only really suitable for building binary RPMs for internal usage. If you want to build public RPMs you should use bdist_rpm only to generate the specfile, and put this together with the package's real source distribution yourself. Using bdist_rpm just to get the specfile is also the 'least moving parts' option if it's giving you a lot of trouble.

Distutils has three ways of figuring out what to put in the step #1 source tarball, documented in more detail in the creating a source distribution section of the Distributing Python Modules documentation. They are, in order of preference:

• a MANIFEST file that lists all files that should be included, or
• a MANIFEST.in file that gives general directions (often also using stuff from setup.py), or
• reverse-engineering the list of sources from setup.py.

If this step is omitting files you want, the big hammer is to just write a MANIFEST file (or modify the one sdist leaves sitting around).

The RPM specfile created in step #2 automatically picks up and packages everything that a normal installation of the package creates. At most, you may find that it's omitting extra documentation or examples or the like that you want packaged up. (It's supposed to automatically package up README files as %doc files, but this doesn't always work.)

Steps #3 and #4 assume a standard RPM subdirectory layout. Unfortunately this explodes if you have used a $HOME/.rpmmacros file to rearrange things more cleanly. The easiest workaround is to use something like 'HOME=/ python setup.py bdist_rpm', so that it doesn't consult your .rpmmacros file.