How to Run a Jupyter Notebook in a Virtual Environment

To help make our interactive data analysis that's in a Jupyter Notebook reproducible, I'd like to encourage you to run it in virtual environments. This helps you keep track of what packages -- and what versions -- were used for a given project. Here are the 4 steps to doing so. 

Step 0: Decide on a Name for the Virtual Environment

Suppose we want to name our virtual environment env-project-a.

Step 1: Create the Virtual Environment

Approach a: Create a virtual environment via the Anaconda distribution of Python

Terminal, do all of the following:

• Create new virtual environment, named env-project-a:

conda create --name env-project-a

• Note: any virtual environment files created for env-project-a will be saved in a path similar to:  /opt/anaconda3/envs/env-project-a

• Start working in the virtual environment:

conda activate env-project-a

• Close virtual environment

conda deactivate

Approach b: Create a virtual environment via the official Python distribution

Terminal, do all of the following:

• Determine what version of Python you have installed (e.g. python3.7)

• Install virtualenvwrapper via pip, for creating and managing virtual environments 

• Specify the complete path to your Python installation, for use with creating virtual environments:

export VIRTUALENVWRAPPER_PYTHON=$(which python3.7)

• Source the script for creating and managing virtual environments 

source /usr/local/bin/virtualenvwrapper.sh

• Navigate to a location that you want to save any virtual environment files for env-project-a  and  create an "Envs" folder for doing so:

export WORKON_HOME=./Envs

• Create the new virtual environment, named env-project-a:

mkvirtualenv env-project-a

• Start working in the virtual environment:

source ./Envs/env-project-a/bin/activate

• To close a virtual environment:

deactivate

Step 2: Use Virtual Environment with your Notebook

• In your active virtual environment, install Jupyter Lab via conda/pip, which will only make Jupyter Lab available to you, when you're in the environment named  env-project-a.

• Set virtual environment for Jupyter notebook

ipython kernel install --user --name=env-project-a

• Start Jupyter Lab

jupyter lab

Step 3: Updating the Virtual Environment in a Notebook

If you need to change the name of the virtual environment previously associated with a Jupyter Notebook, you'll need to edit the notebook's metadata. 

Approach a: Using your favorite text editor

1. Open the notebook with your favorite text editor, and locate the  "kernelspec" dictionary:

2. Update the values associated with both of the  "display_name" and the "name" keys and save the file

3. Follow Step 2 (above) to reopen the notebook in the new virtual environment

Approach b: Directly in the notebook

1. Start the notebook

2. In the Jupyter Notebook Toolbar go to Edit -> Edit Notebook Metadata

3. Follow Steps 2 and 3 of Approach 3a above

Tips

• • One virtual environment per project.

  • requirements.txt that's version controlled via Git, to help you manage and version control the packages you've installed into your virtual environment. Then you can install all the packages into a new virtual environment from command line via:

pip install -r requirements.txt

Keywords: Data products, reproducibility, virtual environments, python, Jupyter Lab, Jupyter notebook, Anaconda, conda, pip, notebook metadata

You may also like: 

• Strategic Plan you Need to get your Data Product into Production 

• More Anaconda commands, related to managing virtual environments