Creating a virtual environment

As a first step when working with Python, it is a good practice to explicitly define the working environment. This helps with detaching from the operative system interpreter and environment, and properly defining the dependencies that will be used. Not doing so tends to generate chaotic scenarios. Remember, explicit is better than implicit!

This is especially important in two scenarios:

• When dealing with multiple projects on the same computer, as they can have different dependencies that clash at some point. For example, two versions of the same module cannot be installed in the same environment.
• When working on a project that will be used on a different computer, for example, developing some code in a personal laptop that will ultimately run in a remote server.

A common joke among developers is responding to a bug with it runs on my machine, meaning that it appears to work on their laptop, but not on the production servers. Although a huge number of factors can produce this error, a good practice is to produce an automatically replicable environment, reducing uncertainty over what dependencies are really being used.

This is easy to achieve using the virtualenv module, which sets up a virtual environment, so none of the installed dependencies will be shared with the Python version installed on the machine.

In Python3, the virtualenv tool is installed automatically, which was not the case in previous versions.

Getting ready

To create a new virtual environment, do the following:

1. Go to the main directory that contains the project.
2. Type the following command:

$ python3 -m venv .venv

This creates a subdirectory called .venv that contains the virtual environment.

The directory containing the virtual environment can be located anywhere. Keeping it on the same root keeps it handy, and adding a dot in front of it avoids it being displayed when running ls or other commands.

3. Before activating the virtual environment, check the version installed in pip. This is different depending on your operative system, for example, 9.0.3 for MacOS High Sierra 10.13.4. It will be upgraded later. Also check the referenced Python interpreter, which will be the main operating system one:

$ pip --version
pip 9.0.3 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)
$ which python3
/usr/local/bin/python3

Now, your virtual environment is ready to go.

How to do it...

1. Activate the virtual environment by running this:

$ source .venv/bin/activate

You'll notice that the prompt will display (.venv), showing that the virtual environment is active.

2. Notice that the Python interpreter used is the one inside the virtual environment, and not the general operative system one from step 3 of Getting ready. Checking the location within a virtual environment:

(.venv) $ which python
/root_dir/.venv/bin/python
(.venv) $ which pip
/root_dir/.venv/bin/pip

3. Upgrade the version of pip and check the version:

(.venv) $ pip install --upgrade pip
...
Successfully installed pip-10.0.1
(.venv) $ pip --version
pip 10.0.1 from /root_dir/.venv/lib/python3.6/site-packages/pip (python 3.6)

4. Get out of the environment and run pip to check the version, which will return the previous environment. Check the pip version and the Python interpreter to show the ones before activating the virtual environment ones, as shown in step 3 of the Getting ready section. Note that they are different pip versions!

(.venv) $ deactivate 
$ which python3
/usr/local/bin/python3
$ pip --version
pip 9.0.3 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)

How it works...

Notice that inside the virtual environment you can use python instead of python3, although python3 is available as well. This will use the Python interpreter defined in the environment.

In some systems like Linux, it's possible that you need to use python3.7 instead of python3. Verify that the Python interpreter you're using is 3.7 or higher.

Inside the virtual environment, step 3 of the How to do it... section installs the most recent version of pip, without affecting the external installation.

The virtual environment contains all the Python data in the .venv directory, and the activate script points all the environment variables there. The best thing about it is that it can be deleted and recreated very easily, removing the fear of experimenting in a self-contained sandbox.

Remember that the directory name is displayed in the prompt. If you need to differentiate the environment, use a descriptive directory name, such as .my_automate_recipe, or use the --prompt option.

There's more...

To remove a virtual environment, deactivate it and remove the directory:

(.venv) $ deactivate
$ rm -rf .venv

The venv module has more options, which can be shown with the -h flag:

$ python3 -m venv -h
usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
 [--upgrade] [--without-pip] [--prompt PROMPT]
 ENV_DIR [ENV_DIR ...]
Creates virtual Python environments in one or more target directories.
positional arguments:
 ENV_DIR A directory to create the environment in.

optional arguments:
 -h, --help show this help message and exit
 --system-site-packages
 Give the virtual environment access to the system
 site-packages dir.
 --symlinks Try to use symlinks rather than copies, when symlinks
 are not the default for the platform.
 --copies Try to use copies rather than symlinks, even when
 symlinks are the default for the platform.
 --clear Delete the contents of the environment directory if it
 already exists, before environment creation.
 --upgrade Upgrade the environment directory to use this version
 of Python, assuming Python has been upgraded in-place.
 --without-pip Skips installing or upgrading pip in the virtual
 environment (pip is bootstrapped by default)
 --prompt PROMPT Provides an alternative prompt prefix for this
 environment.
Once an environment has been created, you may wish to activate it, for example, by
sourcing an activate script in its bin directory.

A convenient way of dealing with virtual environments, especially if you often have to swap between them, is using the virtualenvwrapper module:

1. To install it, run this:

$ pip install virtualenvwrapper

2. Then, add the following variables to your sheet startup script, these normally being .bashrc or .bash_profile. The virtual environments will be installed under the WORKON_HOME directory instead of the same directory as the project, as shown previously:

export WORKON_HOME=~/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh

Sourcing the startup script or opening a new Terminal will allow you to create new virtual environments:

$ mkvirtualenv automation_cookbook
...
Installing setuptools, pip, wheel...done.
(automation_cookbook) $ deactivate
$ workon automation_cookbook
(automation_cookbook) $

For more information, check the documentation of virtualenvwrapper at: https://virtualenvwrapper.readthedocs.io/en/latest/index.html.

Hitting the Tab key after workon autocompletes with the available environments.

See also

• The Installing third-party packages recipe
• The Using a third-party tool—parse recipe