Wandering Thoughts archives

Python virtual environments can usually or often be moved around

Python virtual environments are magical in various ways. They get transparently added to sys.path and programs can be outside of them as long as they use the venv's Python (which is normally a symlink to some system version of Python), for two examples. All of this magic is triggered by the presence of a pyvenv.cfg file at the root of the venv (cf). The contents of this pyvenv.cfg are very minimal and in particular they don't name the location of the venv's root.

For example, here's a pyvenv.cfg:

home = /usr/bin
include-system-site-packages = false
version = 3.10.5

In fact this is the pyvenv.cfg of no less than six venvs from my Fedora 36 desktop (these are being managed through pipx, but pipx creates normal venvs). All of the pyvenv.cfg files all have the same contents because they're all using the same Python version and general settings.

Since pyvenv.cfg and the rest of the virtual environment don't contain any absolute paths to themselves and so don't 'know' where they're supposed to be, it's possible to move venvs around on the filesystem. As a corollary of this it's possible to copy a venv to a different system (in a different filesystem location or the same), provided that the system has the same version of Python, which is often the case if you're using the same Linux distribution version on both. This doesn't seem to be explicitly documented in the venv module documentation and it's possible that some Python modules you may install do require absolute paths and aren't movable, but it seems to be generally true.

If you use pipx there's a caution here, because pipx writes a pipx_shared.pth file into the venv's site-packages directory that does contain the absolute path to its shared collection of Python stuff. I believe this is part of the underlying cause of pipx's problem with Python version upgrades, which is fixed by removing this shared area and having pipx rebuild it.

Another caution comes from systems like Django which may create standard programs or files as part of their project setup. If you create a Django venv and start a Django project from it, Django will create a 'manage.py' executable for the project that has the venv's (current) absolute path to its Python interpreter burned into its '#!' line. If you then move this venv, your manage.py will (still) try to use the Python from the old venv's location, which will either not work or get you the wrong site-packages path.

On the one hand, it's convenient that this works in general, and that there's nothing in the general design of virtual environments that blocks it. On the other hand, it's clear that you can have various corner cases (as shown with pipx and Django), so it's probably best to create your venvs in their final location if you can. If you do have to move venvs (for example they have to be built in one directory and deployed under another), you probably want to test the result and scan for things with the absolute path burned into them.

(I noticed this pyvenv.cfg behavior when I first looked at venvs and sys.path, but I didn't look very much into it at the time. As usual, writing an entry about this has left me better informed than before I started.)