Python on MacOS: Best Practices

Okay, best for me, anyway. In some ways, setting up Python isn’t terribly hard. You can just download an installer from Python.org and be done with it. Or download Anaconda or another distribution like it. But there are a ton of reasons that might not work for you, and doesn’t for me.

My Go-To Toolchain

Updated python, virtualenvwrapper, and VSCode. That’s the name of the game. For the last year and a half or so I’ve been mostly developing on Windows, but I’m back on a Mac and installing a development environment there for the first time in a while, and there are some oddities to work through there.

1 – Python

First you’ve got to get an up-to-date python 2.7, and possibly 3.x as well. MacOS always ships with an old python installed in /usr/bin – as of this writing, Mac OS 10.13.6 is the latest, with Python 2.7.10. My preferred way to do this is:

1. Install homebrew: https://brew.sh/
2. brew install python@2
3. brew install python

This will get you a shiny new up-to-date python at /usr/local/bin, as well as a python3. If you want, you can just install python3 and point the /usr/local/bin/python symlink at that, but I think it’s probably not ideal – it might break other things on the system that expect python to mean 2.7.

This is soft of where virtualenvs come in – make python behave just as you like, on a project-by-project basis

2 – virtualenvwrapper

Using python virtual environments gives a few advantages over using your system environment for everything. Most obviously, you can install only the packages your project needs, and keep track of what that list is. You can also get specific about the versions of those packages, or even of the python interpreter itself. This last bit is really useful – you don’t want to override python with python3 at the system level, but maybe you want to develop a project with python 3.x and don’t want to manually remember to use python3 for everything.

Installing virtualenvwrapper is easy:

1. pip install virtualenvwrapper

Using it is mostly a matter of familiarizing yourself: http://virtualenvwrapper.readthedocs.io/en/latest/

But there’s a bit of a hitch here. It turns out, there are two kinds of Python on Mac – “framework builds” and not-that. The key differentiation here is that framework builds have access to GUI APIs, which is necessary for all softs of graphical things in Python. Most importantly for me, matplotlib doesn’t work outside of a framework build. Conveniently, homebrew installs framework builds by default now. Inconveniently, virtualenv doesn’t, whether or not the source binary is a framework build. So even if your system python is a framework build, the one inside my_env when you mkvirtualenv my_env won’t be.

There’s all sorts of relevant info in this thread, but this comment in particular highlights a so-far very functional solution copied below: https://github.com/pypa/virtualenv/issues/54#issuecomment-177344059

$ # install the utility
$ pip install git+https://github.com/gldnspud/virtualenv-pythonw-osx.git
$ # enter the virtualenv with virtualenvwrapper (or manually)
$ workon my-venv
$ # double-check that this is your venv Python binary
$ which python
/Users/macbook/.virtualenvs/my-venv/bin/python
$ # fix it, using magic
$ fix-osx-virtualenv `which python`/../..

Of course, this has to be done every time a new virtualenv gets made, so to make it automatic, here’s the ~/.virtualenvs/postmkvirtualenv I’m using:

#!/bin/bash
# This hook is sourced after a new virtualenv is activated.

if [ ! -d $VIRTUAL_ENV/Python.app ]; then
  echo Fixing OSX Python display issues...
  fix-osx-virtualenv $VIRTUAL_ENV
fi

With this little hack/workaround, your not-framework-build is magically framework-ized and all GUI-requiring packages in your virtualenv will work as expected. I’m unaware of any drawbacks yet, aside from the up-front hassle.

VSCode

The last piece of this is VSCode. My typical workflow is to have the code command available on my terminal (Shift-Cmd-P>Install code command, inside vscode). When I want to work on a project, I go there in terminal, workon my_env, then code .. That is:

$ mkdir my_proj
$ cd my_proj
$ mkvirtualenv -p python3 my_env_3
$ workon my_env_3
(my_env_3) $ code .

This will launch VSCode with your virtualenv at the front of the $PATH, adopting the virtualenv’s interpreter and package list.

But there’s a bug at least as of time of writing – newly-created terminals and child processes in VSCode get created with the virtualenv at the END of the $PATH, for some reason that totally escapes me. Here’s the bug report: https://github.com/Microsoft/vscode-python/issues/2333#issuecomment-410426411

Luckily, the workaround isn’t terrible – every time you launch code with a new virtualenv, instead of it just working, you have to make the extra step of Shift-Cmd-P>Python: Select interpreter, and pick the interpreter inside your virtualenv. And make sure you pick the one with Python.app in its path, so that you’re getting the framework build that works with GUI APIs.

Another possible workaround is to put the following in your ~/.bash_profile:

# This shenanigans activates orphaned virtual environments in VSCode.
# If VIRTUAL_ENV is set, you know that you're in a child shell launched from inside a virtualenv.
# I thought about just adding it back to the front of the path, but decided it'd be cleaner to do
# a full <workon the_env> just in case.
if [ -z ${VIRTUAL_ENV+x} ]; then
    :
    #echo "VIRTUAL_ENV is unset”
else
    # VIRTUAL_ENV is set to something.
    if [[ $PATH == ${VIRTUAL_ENV}* ]]; then
        # VIRTUAL_ENV is set but also is already at the beginning of the PATH
        :
    else
        echo VIRTUAL_ENV is set to $VIRTUAL_ENV, but is not at start of PATH. Reactivating.
        workon `basename $VIRTUAL_ENV`
        #PATH="${VIRTUAL_ENV}:${PATH}”
    fi
fi

By adding that, you should just about completely obviate the need for setting your python interpreter deliberately in settings.json as we do above, and bonus, all child terminals in Code will have the proper environment activated so you’ll never accidentally pip install something into the wrong environment!

Pylint for Python 3.7.0

One last note: the latest Pylint release doesn’t properly support Python 3.7.0 at this time. The workaround is to install the latest beta release with pip install --pre -U pylint astroid. This will fix the RuntimeError: generator raised StopIteration in Code when it automatically runs the linter and that fails. If you are in the habit of creating lots of virtualenvs and this is a big annoyance, you might add the line above into $WORKON_HOME/postmkvirtualenv as well, but be sure you set a reminder to remove it once the latest Pylint stable release supports 3.7.0 properly.

And that ought to do it!

Footnote: if you’re interested in my process for figuring out all the bugs with this setup, here’s an excel sheet of experiment data: