Activating a virtual environment

As a first step when working with Python, it is a good practice to explicitly define the working environment.

This helps you to detach from the operating system interpreter and environment and properly define the dependencies that will be used. Not doing so tends to generate chaotic scenarios. Remember, explicit is better than implicit!

Explicit is better than implicit is one of the most quoted parts of the Zen of Python. The Zen of Python is a list of general guidelines for Python, to provide clarity on what is considered Pythonic. The full Zen of Python can be invoked from the Python interpreter by calling import this.

This is especially important in two scenarios:

• When dealing with multiple projects on the same computer, as they can have different dependencies that clash at some point. For example, two versions of the same module cannot be installed in the same environment.
• When working on a project that will be used on a different computer, for example, developing some code on a personal laptop that will ultimately run in a remote server.

A common joke among developers is responding to a bug with "it runs on my machine," meaning that it appears to work on their laptop, but not on the production servers. Although a huge number of factors can produce this error, a good practice is to produce an automatically replicable environment, reducing uncertainty over what dependencies are really being used.

This is easy to achieve using the venv module, which sets up a local virtual environment. None of the installed dependencies will be shared with the Python interpreter installed on the machine, creating an isolated environment.

In Python 3, the venv tool is installed as part of the standard library. This was not the case in the previous version where you had to install the external virtualenv package.

Getting ready

To create a new virtual environment, do the following:

1. Go to the main directory that contains the project:

   $ cd my-directory
   

2. Type the following command:

   $ python3 -m venv .venv
   

   This creates a subdirectory called .venv that contains the virtual environment.

   The directory containing the virtual environment can be located anywhere. Keeping it on the same root keeps it handy, and adding a dot in front of it avoids it being displayed when running ls or other commands.

3. Before activating the virtual environment, check the version installed in pip. This is different depending on your operating system and installed packages. It may be upgraded later. Also, check the referenced Python interpreter, which will be the main operating system one:

   $ pip --version
   pip 10.0.1 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
   $ which python3
   /usr/local/bin/python3
   

Note that which may not be available in your shell. In Windows, for example, where can be used.

Now your virtual environment is ready to go.

How to do it…

1. Activate the virtual environment if you use Linux or macOS by running:

   $ source .venv/bin/activate
   

   Depending on your operating system (for example, Windows) and shell (for example, fish), you may need a different command. View the documentation of venv in the Python documentation here: https://docs.python.org/3/library/venv.html.

   You'll notice that the shell prompt will display (.venv), showing that the virtual environment is active.

2. Notice that the Python interpreter used is the one inside the virtual environment, and not the general operating system one from step 3 of the Getting ready section. Check the location within a virtual environment:

   (.venv) $ which python
   /root_dir/.venv/bin/python
   (.venv) $ which pip
   /root_dir/.venv/bin/pip
   

3. Upgrade the version of pip and then check the version:

   (.venv) $ pip install --upgrade pip
   ...
   Successfully installed pip-20.0.2
   (.venv) $ pip --version
   pip 20.0.2 from /root_dir/.venv/lib/python3.8/site-packages/pip (python 3.8)
   

   An alternative is to run python -m ensurepip -U, which will ensure that pip is installed.

4. Get out of the environment and run pip to check the version, which will return the previous environment. Check the pip version and the Python interpreter to show the existing directories before activating the virtual environment directories, as shown in step 3 of the Getting ready section. Note that they are different pip versions:

   (.venv) $ deactivate
   $ which python3
   /usr/local/bin/python3
   $ pip --version
   pip 10.0.1 from /usr/local/lib/python3.8/site-packages/pip (python 3.8)
   

How it works…

Notice that inside the virtual environment you can use python instead of python3, although python3 is available as well. This will use the Python interpreter defined in the environment.

In some systems, like Linux, it's possible that you'll need to use python3.8 instead of python3. Verify that the Python interpreter you're using is 3.8 or higher.

Inside the virtual environment, step 3 of the How to do it section installs the most recent version of pip, without affecting the external installation.

The virtual environment contains all the Python data in the .venv directory, and the activate script points all the environment variables there. The best thing about it is that it can be deleted and recreated very easily, removing the fear of experimenting in a self-contained sandbox.

Remember that the directory name is displayed in the prompt. If you need to differentiate the environment, use a descriptive directory name, such as .my_automate_recipe, or use the –prompt option.

There's more…

To remove a virtual environment, deactivate it and remove the directory:

(.venv) $ deactivate
$ rm -rf .venv
The venv module has more options, which can be shown with the -h flag:
$ python3 -m venv -h
usage: venv [-h] [--system-site-packages]
            [--symlinks | --copies] [--clear]
            [--upgrade] [--without-pip]
            [--prompt PROMPT]
            ENV_DIR [ENV_DIR ...]
Creates virtual Python environments in one or more target directories.
positional arguments:
  ENV_DIR               A directory to create the
                        environment in.
optional arguments:
  -h, --help            show this help message and
                        exit
  --system-site-packages
                        Give the virtual
                        environment access to the
                        system site-packages dir.
  --symlinks            Try to use symlinks rather
                        than copies, when symlinks
                        are not the default for the
                        platform.
  --copies              Try to use copies rather
                        than symlinks, even when
                        symlinks are the default
                        for the platform.
  --clear               Delete the contents of the
                        environment directory if it
                        already exists, before
                        environment creation.
  --upgrade             Upgrade the environment
                        directory to use this
                        version of Python, assuming
                        Python has been upgraded
                        in-place.
  --without-pip         Skips installing or
                        upgrading pip in the
                        virtual environment (pip is
                        bootstrapped by default)
  --prompt PROMPT       Provides an alternative
                        prompt prefix for this
                        environment.
Once an environment has been created, you may wish
to activate it, e.g. by sourcing an activate script
in its bin directory.

A convenient way of dealing with virtual environments, especially if you often have to swap between them, is to use the virtualenvwrapper module:

1. To install it, run this:

   $ pip install virtualenvwrapper
   

2. Then, add the following variables to your shell startup script, these are normally .bashrc or .bash_profile. The virtual environments will be installed under the WORKON_HOME directory instead of the same directory as the project, as shown previously:

   export WORKON_HOME=~/.virtualenvs
   source /usr/local/bin/virtualenvwrapper.sh
   

Sourcing the startup script or opening a new terminal will allow you to create new virtual environments:

$ mkvirtualenv automation_cookbook
...
Installing setuptools, pip, wheel...done.
(automation_cookbook) $ deactivate
$ workon automation_cookbook
(automation_cookbook) $

For more information, view the documentation of virtualenvwrapper at https://virtualenvwrapper.readthedocs.io/en/latest/index.html.

An alternative tool for defining environments is Poetry (https://python-poetry.org/). This tool is designed for creating consistent environments with clear dependencies, and provides commands for upgrades and managing dependency packages. Check it out to see whether it's useful in your use case.

Hitting the Tab key after workon autocompletes the command with the available environments.

See also

• The Installing third-party packages recipe, covered later in the chapter.
• The Using a third-party tool—parse recipe, covered later in the chapter.