Building a gesture-based interface with PyAutoGUI

Now that we can recognize the hand gestures with the Raspberry Pi Pico, our final objective is to create a touchless interface for controlling YouTube video playback.

In this recipe, we will develop a Python script to read the recognized motion transmitted over the serial port and use the PyAutoGUI library to build a gesture-based interface to play, pause, mute, unmute, and change YouTube videos.

Getting ready

The Python script that we are going to develop in this recipe relies primarily on two Python libraries:

• pySerial, which allows us to read the data transmitted serially
• PyAutoGUI (https://pyautogui.readthedocs.io/), which allows controlling the mouse and simulating keyboard input

The PyAutoGUI library can be installed with the following pip command:

$ pip install pyautogui

The API of this PyAutoGUI has been designed to be very simple to use. For example, we can easily...