Creating a virtual environment with Python 3.x and PEP 405

Throughout this book, we will be working with different packages and libraries to create RESTful Web Services, and therefore it is convenient to work with Python virtual environments. Python 3.3 introduced lightweight virtual environments and they were improved in Python 3.4. We will work with these virtual environments, and therefore you will need Python 3.4 or greater. You can read more information about PEP 405 Python Virtual Environment, that introduced the venv module, here: https://www.python.org/dev/peps/pep-0405. All the examples in this book were tested on Python 3.6.2 on Linux, macOS, and Windows.

In case you decide to use the popular virtualenv (https://pypi.python.org/pypi/virtualenv) third-party virtual environment builder or the virtual environment options provided by your Python IDE, you just have to make sure that you activate your virtual environment with the appropriate mechanism whenever it is necessary to do so, instead of following the step explained to activate the virtual environment generated with the venv module integrated in Python.

Each virtual environment we create with venv is an isolated environment and it will have its own independent set of installed Python packages in its site directories (folders). When we create a virtual environment with venv in Python 3.4 and greater, pip is included in the new virtual environment. In Python 3.3, it was necessary to manually install pip after creating the virtual environment. Note that the instructions provided are compatible with Python 3.4 or greater, including Python 3.6.2.

In order to create a lightweight virtual environment, the first step is to select the target folder or directory for it. The following is the path we will use in the example for Linux and macOS.

The target folder for the virtual environment will be the HillarDjangoREST/01 folder within our home directory. For example, if our home directory in macOS or Linux is /Users/gaston, the virtual environment will be created within /Users/gaston/HillarDjangoREST/01. You can replace the specified path with your desired path in each command:

    ~/HillarDjangoREST/01

The following is the path we will use in the example for Windows. The target folder for the virtual environment will be the HillarDjangoREST\01 folder within our user profile folder. For example, if our user profile folder is C:\Users\gaston, the virtual environment will be created within C:\Users\gaston\HillarDjangoREST\01. You can replace the specified path with your desired path in each command:

    %USERPROFILE%\HillarDjangoREST\01

In Windows PowerShell, the previous path would be as follows:

    $env:userprofile\HillarDjangoREST\01

Now, we will create a new virtual environment with venv. In order to do so, we have to use the -m option followed by the venv module name and the desired path to make Python run this module as a script and create a virtual environment in the specified path. The instructions are different depending on the platform in which we are creating the virtual environment.

Open Terminal in Linux or macOS and execute the following command to create a virtual environment:

    python3 -m venv ~/HillarDjangoREST/01

In Windows, in Command Prompt, execute the following command to create a virtual environment:

    python -m venv %USERPROFILE%\HillarDjangoREST\01

If you want to work with Windows PowerShell, execute the following command to create a virtual environment:

    python -m venv $env:userprofile\HillarDjangoREST\01

None of the previous commands produce any output. The script created the specified target folder and installed pip by invoking ensurepip because we didn't specify the --without-pip option.

Understanding the directory structure for a virtual environment

The specified target folder has a new directory tree that contains Python executable files and other files that indicate it is a PEP405 virtual environment.

In the root directory for the virtual environment, the pyenv.cfg configuration file specifies different options for the virtual environment and its existence is an indicator that we are in the root folder for a virtual environment. In Linux and macOS, the folder will have the following main subfolders: bin, include, lib, lib/python3.6, and lib/python3.6/site-packages. In Windows, the folder will have the following main subfolders: Include, Lib, Lib\site-packages, and Scripts. The directory trees for the virtual environment in each platform are the same as the layout of the Python installation on these platforms.

The following diagram shows the folders and files in the directory trees generated for the 01 virtual environments in macOS and Linux platforms:

The following diagram shows the main folders in the directory trees generated for the virtual environment in Windows:

After we activate the virtual environment, we will install third-party packages into the virtual environment and the modules will be located in the lib/python3.6/site-packages or Lib\site-packages folder, based on the platform. The executables will be copied in the bin or Scripts folder, based on the platform. The packages we install won't make changes to other virtual environments or our base Python environment.

Activating the virtual environment

Now that we have created a virtual environment, we will run a platform-specific script to activate it. After we activate the virtual environment, we will install packages that will only be available in this virtual environment. This way, we will work with an isolated environment in which all the packages we install won't affect our main Python environment.

Note that the results of this command will be accurate if you don't start a different shell than the default shell in the terminal session. If you have doubts, check your terminal configuration and preferences. Run the following command in the Terminal in Linux or macOS:

    echo $SHELL

The command will display the name of the shell you are using in the Terminal. In macOS, the default is /bin/bash and this means you are working with the bash shell. Depending on the shell, you must run a different command to activate the virtual environment in Linux or macOS.

If your Terminal is configured to use the bash shell in Linux or macOS, run the following command to activate the virtual environment. The command also works for the zsh shell:

    source ~/HillarDjangoREST/01/bin/activate

If your Terminal is configured to use either the csh or tcsh shell, run the following command to activate the virtual environment:

    source ~/HillarDjangoREST/01/bin/activate.csh

If your Terminal is configured to use the fish shell, run the following command to activate the virtual environment:

    source ~/HillarDjangoREST/01/bin/activate.fish

After you activate the virtual environment, Command Prompt will display the virtual environment root folder name enclosed in parentheses as a prefix of the default prompt to remind us that we are working in the virtual environment. In this case, we will see (01) as a prefix for the Command Prompt because the root folder for the activated virtual environment is 01.

The following screenshot shows the virtual environment activated in a macOS Sierra Terminal with a bash shell, after executing the previously shown commands:

As we can see from the previous screenshot, the prompt changed from

Gastons-MacBook-Pro:~ gaston$ to (01) Gastons-MacBook-Pro:~ gaston$ after the activation of the virtual environment.

In Windows, you can run either a batch file in the Command Prompt or a Windows PowerShell script to activate the virtual environment.

If you prefer Command Prompt, run the following command in the Windows command line to activate the virtual environment:

    %USERPROFILE%\HillarDjangoREST\01\Scripts\activate.bat

The following screenshot shows the virtual environment activated in Windows 10 Command Prompt, after executing the previously shown commands:

As we can see from the previous screenshot, the prompt changed from C:\Users\gaston to (01) C:\Users\gaston after the activation of the virtual environment.

If you prefer Windows PowerShell, launch it and run the following commands to activate the virtual environment. Note that you must have scripts execution enabled in Windows PowerShell to be able to run the script:

    cd $env:USERPROFILE
    HillarDjangoREST\01\Scripts\Activate.ps1

If you receive an error similar to the following lines, it means that you don't have scripts execution enabled:

    C:\Users\gaston\HillarDjangoREST\01\Scripts\Activate.ps1 : File C:\Users\gaston\HillarDjangoREST\01\Scripts\Activate.ps1 cannot be loaded because running scripts is disabled on this system. For more information, see about_Execution_Policies at
    http://go.microsoft.com/fwlink/?LinkID=135170.
    At line:1 char:1
    + C:\Users\gaston\HillarDjangoREST\01\Scripts\Activate.ps1
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : SecurityError: (:) [], PSSecurityException
        + FullyQualifiedErrorId : UnauthorizedAccess
  

The Windows PowerShell default execution policy is Restricted. This policy allows the execution of individual commands but it doesn't run scripts. Thus, in case you want to work with Windows PowerShell, you will have to change the policy to allow the execution of scripts. It is very important to make sure that you understand the risks of the Windows PowerShell execution policies that allow you to run unsigned scripts. For more information about the different policies, check the following web page: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policies?view=powershell-6.

The following screenshot shows the virtual environment activated in a Windows 10 PowerShell, after executing the previously shown commands:

Deactivating the virtual environment

It is extremely easy to deactivate a virtual environment generated by the previously explained process. The deactivation will remove all the changes made in the environment variables and will change the prompt back to its default message. Once you deactivate a virtual environment, you will go back to the default Python environment.

In macOS or Linux, just type deactivate and press Enter.

In a Windows Command Prompt, you have to run the deactivate.bat batch file included in the Scripts folder. In our example, the full path for this file is %USERPROFILE%\HillarDjangoREST\01\Scripts\deactivate.bat.

In Windows PowerShell, you have to run the Deactivate.ps1 script in the Scripts folder. In our example, the full path for this file is $env:userprofile\HillarDjangoREST\01\Scripts\Deactivate.ps1. Remember that you must have scripts execution enabled in Windows PowerShell to be able to run the script.

The instructions in the next sections assume that the virtual environment we have created is activated.