To perform the hands-0n exercises in this chapter, you will need the following:

• Raspberry Pi 4 Model B
• Raspbian OS Buster (with desktop and recommended software)
• Minimum Python version 3.5

These requirements are what the code examples in this book are based on. It's reasonable to expect that the code examples should work without modification on a Raspberry Pi 3 Model B or a different version of Raspbian OS as long as your Python version is 3.5 or higher.

The full source code for this book can be found on GitHub at the following URL: https://github.com/PacktPublishing/Practical-Python-Programming-for-IoT. We will clone this repository shortly when we come to the Setting up a Python virtual environment section.

In this section, we will find out which versions of Python you have installed on your Raspberry Pi. As we will discover, there are two versions of Python that come pre-installed on Raspbian OS. Unix-based operating systems (such as Raspbian OS) typically have Python version 2 and 3 pre-installed because there are operating-system-level utilities built with Python.

To find out which versions of Python you have on your Raspberry Pi, follow these steps:

1. Open a new Terminal and execute the python --version command:

$ python --version
Python 2.7.16

In my example, we see that Python version 2.7.16 has been installed.

2. Next, run the python3 --version command:

$ python3 --version
Python 3.7.3

In my example, we see that the second version of Python (that is, python3, with the 3) that is installed is version 3.7.3.

Don't worry if the minor versions (the numbers .7.16 after the 2 and .7.3 after 3) are not the same; it is the major versions 2 and 3 that are of interest. Python 2 is a legacy version of Python, while Python 3 is the current and supported version of Python at the time of writing. When we are starting a new Python development, we will practically always use Python 3 unless there are legacy issues we need to contend with.

If you are an experienced Python programmer, you may be able to discern whether a script is written for Python 2 or 3, but it's not always obvious by simply looking at a piece of code. Many new-to-Python developers experience frustrations by mixing up Python programs and code fragments that are meant for different Python versions. Always remember that code written for Python 2 is not guaranteed to be upward-comparable with Python 3 without modification.

A quick tip I can share to visually help to determine which Python version a code fragment is written for (if the programmer has not made it clear in the code comments) is to look for a print statement.

If you look at the following example, you will see that there are two print statements. The first print statement without the parentheses is a give-away that it will only work with Python 2:

print "Hello"  # No parentheses - This only works in Python 2, a dead give-away that this script is for Python 2.

print("Hello") # With parentheses - this will work in Python 2 and Python 3

Of course, you can always run the code against both Python 2 and 3 and see what happens.

We have now seen that there are two Python versions available by default on Raspbian OS, and made mention that there are system-level utilities that are written in Python that reply on these versions. As Python developers, we must take care not to disrupt the global Python installations as this can potentially break system-level utilities.

We will now turn our attention to a very important Python concept, the Python virtual environment, which is the way we isolate or sandbox our own Python projects from the global installation.

In this section, we will discuss how Python interacts with your operating system installation and cover the steps necessary to set up and configure a Python development environment. In addition, as part of our setup process, we will clone the GitHub repository that contains all of the code (organized by chapter) for this book.

By default, Python and its package management tool, pip, operate globally at the system level and can create some confusion for Python beginners because this global default is in contrast to many other language ecosystems that operate locally on a project folder level by default. Unwearyingly working and making changes to the global Python environment can break Python-based system-level tools, and remedying the situation can become a major headache.

As a Python developer, we use Python virtual environments to sandbox our Python projects so they will not adversely interfere with system-level Python utilities or other Python projects.

In this book, we will be using a virtual environment tool known as venv, which comes bundled as a built-in module with Python 3.3 and above. There are other virtual environment tools around, all with their relative strengths and weaknesses, but they all share the common goal of keeping Python dependencies isolated to a project.

Let's begin and clone the GitHub repository and create a new Python virtual environment for this chapter's source code. Open a new Terminal window and work through the following steps:

1. Change into or create a folder where you want to store this book's source code and execute the following commands. With the last command, we rename the cloned folder to be pyiot. This has been done to help shorten Terminal command examples throughout the book:

$ cd ~
$ git clone https://github.com/PacktPublishing/Practical-Python-Programming-for-IoT
$ mv Practical-Python-Programming-for-IoT pyiot

$ cd ~/pyiot/chapter01

3. Execute the following command, which creates a new Python virtual environment using the venv tool. It's important that you type python3 (with the 3) and remember that venv is only available with Python 3.3 and above:

$ python3 -m venv venv

The options that we are passing to python3 include -m venv, which tells the Python interpreter that we want to run the module named venv. The venv parameter is the name of the folder where your virtual environment will be created.

4. To use a Python virtual environment, we must activate it, which is accomplished with the activate command:

# From with in the folder ~/pyiot/chapter01
$ source venv/bin/activate
(venv) $

When your Terminal has a Python virtual environment activated, all Python-related activity is sandboxed to your virtual environment.

5. Next, execute which python (without the 3) in your Terminal, and notice that the location of the Python executable is beneath your venv folder and if you check the version of Python, it's Python version 3:

(venv) $ which python
/home/pi/pyiot/chapter01/venv/bin/python

(venv) $ python --version
Python 3.7.3

6. To leave an activated virtual environment, use the deactivate command as illustrated here:

(venv) $ deactivate
$ 

Notice also that (venv) $ is no longer part of the Terminal prompt text once the virtual environment has been deactivated.

7. Finally, now that you are outside of our Python virtual environment if you execute which python (without the 3) and python --version again, notice we're back to the default system-level Python interpreter, which is version 2:

$ which python
/usr/bin/python

$ python --version
Python 2.7.13

As we just illustrated in the preceding examples, when we ran python --version in an activated virtual environment, we see that it's Python version 3 whereas in the last example, at the start of this chapter, the system level, python --version, was version 2, and we needed to type python3 --version for version 3. In practice, python (with no number) relates to the default version of Python. Globally, this is version 2. In your virtual environment, we only have one version of Python, which is version 3, so it becomes the default.

We have now seen how to create, activate, and deactivate a Python virtual environment and why it is important to use a virtual environment to sandbox Python projects. This sandboxing means we can isolate our own Python projects and their library dependencies from one another, and it prevents us from potentially disrupting the system-level installation of Python and breaking any system-level tools and utilities that rely on them.

Next, we will see how to install and manage Python packages in a virtual environment using pip.

In this section, we learn how to install and manage Python packages in a Python virtual environment you created and explored in the previous section. A Python package (or library if you prefer that term) allows us to extend the core Python language with new features and functionality. 

We will need to install many different packages throughout this book, however, for starters and to explore and learn the basic concepts related to package installation and management, we will be installing two common GPIO-related packages in this section that we will use throughout this book. These two packages are the following:

• The GPIOZero library, an entry-level and easy to use GPIO library for controlling simple electronics
• The PiGPIO library, an advanced GPIO library with many features for more complex electronic interfacing

In the Python ecosystem, package management is done with the pip command (pip stands for Python installs packages). The official public package repository that pip queries is known as the Python Package Index, or simply PyPi, and it is available for browsing on the web at https://pypi.org.

There will be code examples in this book where we will be interacting with your Raspberry Pi's GPIO pins, so we need to install a Python package (or two) so that your Python code can work with your Raspberry Pi's GPIO pins. For now, we are just going to check for and install two GPIO-related packages. In Chapter 2, Getting Started with Python and IoT, and Chapter 5, Connecting Your Raspberry Pi to the Physical World, we will cover these GPIO packages and other alternatives in greater detail.

In your chapter01 source code folder, you will find a file named gpio_pkg_check.py, which is replicated in the following. We will use this file as the basis to learn about pip and package management in the context of a Python virtual environment. This script simply reports the availability of a Python package depending on whether using import succeeds or raises an exception:

"""
Source File: chapter01/gpio_pkg_check.py
"""
try:
    import gpiozero
    print('GPIOZero Available')
except:
    print('GPIOZero Unavailable. Install with "pip install gpiozero"')

try:
    import pigpio
    print('pigpio Available')
except:
    print('pigpio Unavailable. Install with "pip install pigpio"')

Let's check for the availability of GPIO packages using gpio_pkg_check.py and with pip. I'll kill the suspense by telling you that they're not going to be available in your freshly-created virtual environment (yet), however, we are going to install them!

The following steps will walk us through the process of upgrading pip, exploring the tool's options, and installing packages:

1. As the first step, we will upgrade the pip tool. In a Terminal window, run the following command, remembering that all commands that follow must be performed in an activated virtual environment—meaning you should see the text (venv) in the Terminal prompt:

(venv) $ pip install --upgrade pip
...output truncated...

The preceding upgrade command may take a minute or two complete and will potentially output a lot of text to the Terminal.

2. With pip now upgraded, we can see what Python packages are already installed in our virtual environment using the pip list command:

(venv) $ pip list
pip (9.0.1)
pkg-resources (0.0.0)
setuptools (33.1.1)

What we see in the preceding are the default Python packages in our fresh virtual environment. Do not worry if the exact package list or version numbers do not match exactly with the example.

3. Run our Python script with the python gpio_pkg_check.py command and observe that our GPIO packages are not installed:

(venv) $ python gpio_pkg_check.py
GPIOZero Unavailable. Install with "pip install gpiozero"
pigpio Unavailable. Install with "pip install pigpio"

4. To install our two required GPIO packages, we use the pip install command as shown in the following example:

(venv) $ pip install gpiozero pigpio
Collecting gpiozero...
... output truncated ...

5. Now, run the pip list command again; we will see these new packages are now installed in our virtual environment:

(venv) $ pip list
colorzero (1.1)
gpiozero (1.5.0)   # GPIOZero
pigpio (1.42)      # PiGPIO
pip (9.0.1)
pkg-resources (0.0.0)
setuptools (33.1.1)

You may have noticed that there is a package called colorzero (this is a color manipulation library) that we did not install. gpiozero (version 1.5.0) has a dependency on colorzero, so pip has installed it for us automatically.

6. Re-run python gpio_pkg_check.py and we now see that our Python modules are available for import:

(venv) $ python gpio_pkg_check.py
GPIOZero Available
pigpio Available

Great! We now have a virtual environment with two GPIO packages installed. As you work on Python projects, you will inevitably install more and more packages and want to keep track of them.

7. Take a snapshot of the packages you have previously installed with the pip freeze command:

(venv) $ pip freeze > requirements.txt

The preceding example freezes all installed packages into a file named requirements.txt, which is a common filename to use for this purpose.

8. Look inside the requirements.txt file and you will see all of the Python packages listed together with their version numbers:

(venv) $ cat requirements.txt
colorzero==1.1
gpiozero==1.5.0
pigpio==1.42
pkg-resources==0.0.0

In the future, if you move your Python project to another machine or a new virtual environment, you can use your requirement.txt file to install all of your captured packages in one go using the pip install -r requirements.txt command.

We've now completed the basic installation life cycle of Python packages using pip. Note that whenever you install new packages with pip install, you also need to re-run pip freeze > requirements.txt to capture the new packages and their dependencies.

To finish our exploration of pip and package management, here are a few other common pip commands:

# Remove a package
(venv) $ pip uninstall <package name>
 
# Search PyPi for a package (or point your web browser at https://pypi.org)
(venv) $ pip search <query text>
 
# See all pip commands and options (also see Further Reading at the end of the chapter).
(venv) $ pip --help

Congratulations! We've reached a milestone and covered the essential virtual environment principles that you can use for any Python project, even ones that are not Raspberry Pi related! 

Now that we have seen how to create a virtual environment and install packages, let's take a look at a typical Python project folder structure such as ~/pyiot/chapter01 and discover what lies beneath the venv folder.

This section relates to venv, which we have been using in this chapter, and will apply to virtualenv but not pipenv, which we listed as alternative virtual environment tools. The example is also specific to a Raspbian OS and is typical of a standard Unix-based OS. It's important to, at a minimum, understand the basic structure of a virtual environment deployment since we will be mixing our own Python programming code in with the files and folders that make up the virtual environment.

Here is the folder structure of our virtual environment. Yep, its a screenshot from a Mac. That's so I could get everything on screen at once:

The following points explain the core subfolders that are found within our ~/pyiot/chapter01 folder after we ran python3 -m venv venv and installed packages using pip:

• The venv folder contains all of the Python virtual environment files. There is no real practical need to be touching anything under this folder manually—let the tools do that for you. Remember that the folder is named venv only because that's what we called it when it was created.

• The venv/bin folder contains the Python interpreter (in the venv case, there are symbolic links to the system interpreter) and other core Python tools, including pip.
• Underneath the venv/lib folder are all the sandboxed Python packages for the virtual environment, including the GPIOZero and PiGPIO packages we installed using pip install.
• Our Python source file, gpio_pkg_check.py, is in the top-level folder, ~/pyiot/chapter01; however, you can create sub-folders here to help to organize your code and non-code files.
• Finally, requirements.txt lives by convention in the top project folder.

The virtual environment folder venv does not actually need to be kept in the project folder; however, it's often convenient to have it there for activation with the activate command.

It's important to understand that, as a Python developer, you will be mixing in your own programming code with files and folders that form part of the virtual environment system and that you should be pragmatic when selecting which files and folders are added to your version control system, should you be using one.

This last point is important since the virtual environment system can amount to many megabytes in size (and often many times larger than your program code) that does not need versioning (since we can always recreate the virtual environment as long as we have a requirements.txt file), plus it's host platform-specific (that is, there will be differences between Windows, Mac, and Linux), plus there will be differences between different virtual environment tools (for example, venv versus pipenv). As such, virtual environments are not generally portable in projects that involve many developers working on different computers.

Now that we have briefly explored the file and folders structure and the importance of understanding this structure, we will continue and look at alternative ways of running a script that is sandboxed to a virtual environment.

Let's briefly turn our attention to the alternative ways that we can execute a Python script. As we will learn, choosing the appropriate method is all based around how and from where you intend to start your script and whether your code requires elevated permissions.

The most common way of running a Python script is from within its virtual environment and with the permissions of the currently logged in user. However, there will be scenarios where we need to run a script as the root user or from outside an activated virtual environment.

Here are the ways we will explore:

• Using sudo with virtual environments
• Executing Python scripts outside of their virtual environments
• Running a Python script at boot

Let's start by learning how to run a Python script with root user permissions.