A Python program can be outside of a virtual environment it uses
A while ago I wrote about installing modules to a custom location, and in that entry one reason I said for not
doing this with a virtual environment was that I didn't
want to put the program involved into a virtual environment just
to use some Python modules. Recently I realized that you don't have
to, because of how virtual environments add themselves to
sys.path. As long as you run your program using the
virtual environment's Python, it gets to use all the modules you
installed in the venv. It doesn't matter where the program is
and you don't have to move it from its current location, you just
have to change what '
python' it uses.
The full extended version of this is that if you have your program
set up to run using '
#!/usr/bin/env python3', you can change what
Python and thus what virtual environment you use simply by changing
$PATH that it uses. The downside of this is that you can
accidentally use a different Python than you intended because your
$PATH isn't set up the way you thought it was, although in many
cases this will result in immediate and visible problems because
some modules you expected aren't there.
(One way this might happen is if you run the program using the
system Python because you're starting it with a default
One classical way this can happen is running things from crontab
Another possible use for this, especially in the
$PATH based version,
is assembling a new virtual environment with new, updated versions of
the modules you use in order to test your existing program with them.
You can also use this to switch module versions back and forth in live
usage just by changing the
$PATH your program runs with (or by
repeatedly editing its
#! line, but that's more work).
Realizing this makes me much more likely in the future to just use
virtual environments for third party modules. The one remaining
irritation is that the virtual environment is specific to the
Python version, but there are various ways
of dealing with that. This is one of the cases where I think we're
going to want to use '
pip freeze' (in advance) and then exactly
reproduce our previous install in a new virtual environment. Or
maybe we can get '
python3 -m venv --upgrade <venv-dir>' to work,
although I'm not going to hold my breath on that one.
(A quick test suggests that upgrading the virtual environment doesn't work, at least for going from the Ubuntu 18.04 LTS Python 3 to the Ubuntu 20.04 LTS Python 3. This is more or less what I expected, given what would be involved, so building a new virtual environment from scratch it is. I can't say I'm particularly happy with this limitation of virtual environments, especially given that we always have at least two versions of Python 3 around because we always have two versions of Ubuntu LTS in service.)