How to Run a Jupyter Notebook in a Virtual Environment

To help make our interactive data analysis that's in a Jupyter Notebook reproducible, I'd like to encourage you to run it in virtual environments. This helps you keep track of what packages -- and what versions -- were used for a given project. Here are the 4 steps to doing so.

Step 0: Decide on a Name for the Virtual Environment

Suppose we want to name our virtual environment env-project-a.

Step 1: Create the Virtual Environment

Approach a: Create a virtual environment via the Anaconda distribution of Python

Terminal, do all of the following:

Create new virtual environment, named env-project-a:

conda create --name env-project-a

Note: any virtual environment files created for env-project-a will be saved in a path similar to: /opt/anaconda3/envs/env-project-a
Start working in the virtual environment:

conda activate env-project-a

Close virtual environment

conda deactivate

Approach b: Create a virtual environment via the official Python distribution

Terminal, do all of the following:

Determine what version of Python you have installed (e.g. python3.7)
Install virtualenvwrapper via pip, for creating and managing virtual environments
Specify the complete path to your Python installation, for use with creating virtual environments:

export VIRTUALENVWRAPPER_PYTHON=$(which python3.7)

Source the script for creating and managing virtual environments

source /usr/local/bin/virtualenvwrapper.sh

Navigate to a location that you want to save any virtual environment files for env-project-a and create an "Envs" folder for doing so:

export WORKON_HOME=./Envs

Create the new virtual environment, named env-project-a:

mkvirtualenv env-project-a

Start working in the virtual environment:

source ./Envs/env-project-a/bin/activate

To close a virtual environment:

deactivate

Step 2: Use Virtual Environment with your Notebook

In your active virtual environment, install Jupyter Lab via conda/pip, which will only make Jupyter Lab available to you, when you're in the environment named env-project-a.
Set virtual environment for Jupyter notebook

ipython kernel install --user --name=env-project-a

Start Jupyter Lab

jupyter lab

Step 3: Updating the Virtual Environment in a Notebook

If you need to change the name of the virtual environment previously associated with a Jupyter Notebook, you'll need to edit the notebook's metadata.

Approach a: Using your favorite text editor

Open the notebook with your favorite text editor, and locate the "kernelspec" dictionary:
Update the values associated with both of the "display_name" and the "name" keys and save the file
Follow Step 2 (above) to reopen the notebook in the new virtual environment

Approach b: Directly in the notebook

Start the notebook
In the Jupyter Notebook Toolbar go to Edit -> Edit Notebook Metadata
Follow Steps 2 and 3 of Approach 3a above

Tips

- One virtual environment per project.
- requirements.txt that's version controlled via Git, to help you manage and version control the packages you've installed into your virtual environment. Then you can install all the packages into a new virtual environment from command line via:

pip install -r requirements.txt

Keywords: Data products, reproducibility, virtual environments, python, Jupyter Lab, Jupyter notebook, Anaconda, conda, pip, notebook metadata