I should keep track of what Python packages I install through pip

June 28, 2021

These days I'm increasingly making use of installing Python packages through pip, whether this is into a PyPy environment or with 'pip install --user' for things like python-lsp-server. Having done this for a while, complete with trying to keep up with potential package upgrades, I've come to the conclusion that I should explicitly keep track of what packages I install, recording this in some place I can find it again.

There are two problems (or issues) that push me to this. The first is that as far as I know, Pip doesn't keep track of a distinction between packages that you've asked it to install and the dependencies of those packages. All of the packages show up in 'pip list', and any can show up in 'pip list --outdated'. My understanding is that in the normal, expected use of Pip you'll keep track of this in your project in a requirements file, then use that to build the project's virtualenv. This is not really the model of installing commands, especially commands like python-lsp-server that have install time options.

The second issue is that Pip installed packages are implicitly for a specific version of Python. If you rely on the system Python (instead of your own version) and that version gets upgraded, suddenly 'pip list' will report nothing (and you will in fact have no packages available). At this point you need to somehow recover the list of installed packages and re-install all of them (unless you resort to unclean hacks). Explicitly keeping track of this list in advance is easier than having to dig it out at the time.

Having an explicit list helps in other situations. Perhaps you started out installing all of your tools under CPython, but now you want to see how well they'll work under PyPy. Perhaps you're building a new PyPy based environment with a new version of PyPy and want to start over from scratch. Perhaps you think package versions and dependencies have gotten snarled and you're carrying surplus packages, so you want to delete everything and start over from scratch.

(Starting over from scratch can also be the easiest way to get the best version of dependencies, since the packages you're directly installing may have maximum version constraints that will trip you up if you just directly 'pip install --upgrade ...' dependencies.)

PS: Possibly there's ways to do all of this with Pip today, especially things like 'upgrade this and all of its dependencies to the most recent versions that are acceptable'. I'm not well versed in Pip, since mostly I use it as a program installer.

Written on 28 June 2021.
« Some notes on what's in Linux's /sys/class/net for network interface status
Be careful when matching on Ethernet addresses in systemd-networkd »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Jun 28 00:05:23 2021
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.