How I want to use pip with PyPy to install third party programs

April 23, 2021

After seeing PyPy run a moderate program faster than CPython, I wondered how easy it would be to use PyPy to run pyls (which I use for LSP-based editing for Python in GNU Emacs), since pyls is reasonably big and probably reasonably CPU intensive. Pyls is a third party program that's not packaged by Ubuntu but which is available through PyPI, so normally I install it with 'pip3 install --user ...' so that it goes in $HOME/.local/bin.

I'll start with my conclusion: for PyPy, I want to use a virtualenv to install and manage third party programs (at least for the version and setup of PyPy that Ubuntu packages). I'm not normally much of a fan of virtualenvs for various reasons and I'd avoid them here if I could, but using a virtualenv is less annoyance and more likely to succeed than trying to get a pip for PyPy to coexist with pip for CPython. Perhaps you can make it work with enough sweat, but it's a lot easier to shrug, make a virtualenv or two, and accept 30 MBytes of overhead per virtualenv.

You definitely want one virtualenv for PyPy 2 and a second one for PyPy 3. I think you can put all of the third party commands you want into the single virtualenv rather than having to have one virtualenv for pyls, one for YAPF, and so on. To set up my virtualenvs using the Ubuntu version of PyPy, I followed the PyPy documentation for this:

virtualenv -p /usr/bin/pypy3 $HOME/lib/pypy3-env
virtualenv -p /usr/bin/pypy $HOME/lib/pypy2-env

The Ubuntu packaged version of virtualenv is a Python 3 program, but it still works to set up a Python 2 PyPy virtualenv. This is probably routine to people who're familiar with it, but I'm not.

Once your virtualenvs are set up, you can start installing things with the virtualenv's pip as usual:

$HOME/lib/pypy3-env/bin/pip install python-language-server

(For pyls, you'll need the PyPy development packages to be installed. On Ubuntu 20.04 these are pypy-dev and pypy3-dev.)

You don't need to activate the virtualenv to run commands from it; as I found out earlier, virtual environments transparently add themselves to sys.path. I'm not sure what maintenance you'll need to do to a virtualenv when PyPy changes versions (or changes what version of Python 3 it claims to be). I'll probably get to find out someday.

Even if your system version of Python 2 doesn't package and supply pip (Fedora now doesn't ship it), your virtualenv appears to magically get it and it works. I don't quite know how this works (although I'm sure I could find out if I dug into it), but I'm happy with the result since it's quite convenient.

(Our Ubuntu 18.04 machines have no standard package for PyPy 3, but that's another issue. Perhaps we'll be able to switch our user login machines over to 20.04 this summer.)

Written on 23 April 2021.
« Go 1.17 will allow converting a slice to an array pointer (some of the time)
Why people care about SAS despite not being 'enterprise' »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Apr 23 00:25:53 2021
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.