How to Run a Jupyter Notebook in a Virtual Environment

To help make our interactive data analysis that's in a Jupyter Notebook reproducible, I'd like to encourage you to run it in virtual environments. This helps you keep track of what packages -- and what versions -- were used for a given project. Here are the 4 steps to doing so.

Step 0: Decide on a Name for the Virtual Environment

Suppose we want to name our virtual environment env-project-a.

Step 1: Create the Virtual Environment

Approach a: Create a virtual environment via the Anaconda distribution of Python

Terminal, do all of the following:

  • Create new virtual environment, named env-project-a:

conda create --name env-project-a

  • Note: any virtual environment files created for env-project-a will be saved in a path similar to: /opt/anaconda3/envs/env-project-a

  • Start working in the virtual environment:

conda activate env-project-a

  • Close virtual environment

conda deactivate

Approach b: Create a virtual environment via the official Python distribution

Terminal, do all of the following:

  • Determine what version of Python you have installed (e.g. python3.7)

  • Install virtualenvwrapper via pip, for creating and managing virtual environments

  • Specify the complete path to your Python installation, for use with creating virtual environments:

export VIRTUALENVWRAPPER_PYTHON=$(which python3.7)

  • Source the script for creating and managing virtual environments

source /usr/local/bin/virtualenvwrapper.sh

  • Navigate to a location that you want to save any virtual environment files for env-project-a and create an "Envs" folder for doing so:

export WORKON_HOME=./Envs

  • Create the new virtual environment, named env-project-a:

mkvirtualenv env-project-a

  • Start working in the virtual environment:

source ./Envs/env-project-a/bin/activate

  • To close a virtual environment:

deactivate

Step 2: Use Virtual Environment with your Notebook

  • In your active virtual environment, install Jupyter Lab via conda/pip, which will only make Jupyter Lab available to you, when you're in the environment named env-project-a.

  • Set virtual environment for Jupyter notebook

ipython kernel install --user --name=env-project-a

  • Start Jupyter Lab

jupyter lab

Step 3: Updating the Virtual Environment in a Notebook

If you need to change the name of the virtual environment previously associated with a Jupyter Notebook, you'll need to edit the notebook's metadata.

Approach a: Using your favorite text editor

  1. Open the notebook with your favorite text editor, and locate the "kernelspec" dictionary:

  2. Update the values associated with both of the "display_name" and the "name" keys and save the file

  3. Follow Step 2 (above) to reopen the notebook in the new virtual environment

Approach b: Directly in the notebook

  1. Start the notebook

  2. In the Jupyter Notebook Toolbar go to Edit -> Edit Notebook Metadata

  3. Follow Steps 2 and 3 of Approach 3a above

Tips

    • One virtual environment per project.

    • requirements.txt that's version controlled via Git, to help you manage and version control the packages you've installed into your virtual environment. Then you can install all the packages into a new virtual environment from command line via:

pip install -r requirements.txt

Keywords: Data products, reproducibility, virtual environments, python, Jupyter Lab, Jupyter notebook, Anaconda, conda, pip, notebook metadata

You may also like: