Introduction to tinyML
Tiny machine learning, or, as we will refer to it, tinyML, is a technology that is gaining huge momentum in various fields, due to its ability to enable non-intrusive smartness. tinyML is not new, as it has already facilitated consumer electronics like smart speakers and smartwatches for many years. However, recent advances in hardware and software have made it more accessible and practical than ever. Therefore, it is no longer a niche technology.
There are at least three factors that make tinyML particularly appealing: cost, energy, and privacy.
The first benefit given by this technology is its cost-effectiveness. Devices used in tinyML are typically low-cost, ranging from a few cents to a few dollars in most cases. As a result, it is an affordable technology for businesses and individuals to drive innovation.
The second unique advantage of tinyML is its ability to run ML on low-power platforms.
The overall goal of tinyML is to allow smartness through low-power devices. This feature enables applications to operate on compact batteries such as coin cells or even plants (https://www.youtube.com/watch?v=_xELDU15_oE) for months, contributing to tackling energy challenges sustainably.
Privacy is the other factor that makes tinyML an attractive technology. While the internet provides tremendous opportunities, there is always a concern regarding user data exposure to unauthorized parties. The risks here could concern compromised privacy or personal identity theft to commit fraud, just to name a couple. tinyML can mitigate this issue by running ML algorithms on-device without sending data to the cloud.
As you may have noticed, so far, we have discussed why tinyML has the potential to enable ubiquitous intelligence. However, what is tinyML in practical terms?
What is tinyML?
tinyML encompasses the set of ML and embedded system technologies to enable the creation of intelligent applications for low-power devices. Generally, these devices have limited memory and processing power, but they are equipped with sensors to sense the physical environment and make decisions based on ML algorithms.
In tinyML, ML and the deployment platform are not independent entities but entities that need to know each other at best. Building an ML architecture without considering the target device capabilities will make it challenging to deploy effective applications. On the other hand, designing power-efficient processors to expand the ML capabilities of these devices would be impossible without knowing the software algorithms involved. Therefore, we can only bring tremendous and compelling tinyML applications to life through a delicate balance between software and hardware.
Throughout this book, we will explore tinyML with microcontrollers as target devices. Why microcontrollers, you ask? Well, let’s just say that they are the perfect match for what we want, and in the following subsection, we will tell you why.
Why ML on microcontrollers?
The first and foremost reason for choosing microcontrollers is their popularity in various fields, such as automotives, consumer electronics, kitchen appliances, healthcare, and telecommunications. These devices are present in our day-to-day electronic devices, and with the emergence of the Internet of Things (IoT), their market growth has been exponential.
Already in 2018, the market research company IDC (https://www.idc.com) reported 28.1 billion microcontrollers sold worldwide. Those are impressive numbers, considering that 1.5 billion smartphones and 67.2 million PCs were sold in the same year. Therefore, tinyML is a significant milestone in the evolution of IoT devices, paving the way for the proliferation of intelligent and connected low-power devices.
The other reasons for choosing microcontrollers are their affordability, ease of programming, and ability to run sophisticated ML algorithms, making them suitable for a wide range of applications.
However, these devices are generally connected to the internet in the IoT space. Therefore, if we can transmit data to a trusted cloud service, why can’t we delegate the ML computation to it, given its superior performance? In other words, why do we need to run ML locally?
Why run ML on-device?
In addition to privacy, as discussed earlier, there are two other reasons to run ML locally:
- Reducing latency: Sending data back and forth to and from the cloud is not instant and could affect applications that must respond reliably within a time frame.
- Reducing power consumption: Sending and receiving data to and from the cloud is not power-efficient, even when using low-power communication protocols such as Bluetooth.
The following stacked bar chart shows the power consumption breakdown for the on-board components on the Arduino Nano 33 BLE Sense board, one of the microcontroller boards employed in this book:
Figure 1.1: Power consumption breakdown for the Arduino Nano 33 BLE Sense board
Looking at the power consumption breakdown, we can observe that CPU computation uses less power than Bluetooth communication (14% versus 65%). As a result, it is preferable to compute more and transmit less to mitigate the risk of fast battery drain. Typically, the radio module, such as the one used for Bluetooth or other wireless communications, is the component that needs the most power in embedded devices.
Now that we know the benefits of running ML on these tiny devices, what are the practical opportunities and challenges?
The opportunities and challenges for tinyML
tinyML finds its natural home in applications where low power consumption is a critical requirement, such as when a device must operate with a battery for as long as possible.
If we think about it, we are already surrounded by battery-powered products that use ML under the hood. For example, wearable devices, such as smartwatches and fitness tracking bands, can recognize human activities to track our health goals or detect dangerous situations, such as a fall to the ground.
These products are based on tinyML for all intents and purposes because they need on-device ML on a low-power system to interpret sensor data continuously.
However, the use of battery-powered tinyML applications extends beyond wearable devices. For example, there are scenarios where we might need to monitor an environment to detect hazardous conditions, such as detecting fires to prevent them from spreading across a wide area.
There are unlimited use cases for tinyML, and the ones we briefly introduced are only a few.
However, despite the unlimited potential use cases for tinyML, some critical challenges must be addressed. The most significant challenges arise from the computational perspective of our devices, since they are often limited in memory and processing power. We work on systems with a few kilobytes of RAM and, in some cases, processors with no floating-point arithmetic acceleration. Furthermore, the deployment environment could be unfriendly. For example, environmental factors, such as dust and extreme weather conditions, could interfere during the normal execution of our applications.
As we have touched upon deployment environments briefly, let us delve deeper into them in the following subsection.
Deployment environments for tinyML
A tinyML application could live in both centralized and distributed systems.
In a centralized system, the application does not necessarily need to communicate with other devices. Nowadays, we interact with our smartphones, cameras, drones, and kitchen appliances seamlessly with our voices. For example, detecting the magic words “OK, Google,” “Alexa,” and so on in smart assistants is a tinyML application in every respect. In fact, this application can only run locally on a low-power system for a quick response and minimal power usage.
Usually, centralized tinyML applications aim to trigger more power-hungry functionalities, such as activating a media service.
In a distributed system, the device (that is, the node or sensor node) still performs ML locally but also communicates with nearby devices to achieve a common goal, as shown in Figure 1.2:
Since the nodes are part of a network and typically communicate through wireless technologies, we commonly call the network a wireless sensor network (WSN).
While this scenario may appear to conflict with the power consumption implications of transmitting data, devices may still need to collaborate to obtain meaningful knowledge about their working environment. In fact, specific applications may require a holistic understanding of the distribution of physical quantities, such as temperature, humidity, and soil moisture, rather than knowing the values from a particular node.
For example, consider an application to improve agriculture efficiency. In this scenario, a WSN could assist in identifying areas of the field that require more water than others. In fact, by gathering and analyzing data from multiple nodes across the field, a network can provide a comprehensive understanding of the soil moisture levels, helping farmers reduce their water usage. But that’s not all. Efficient communication protocols are crucial for the network’s lifetime. Therefore, we may think of using tinyML to make them more effective. Since sending raw data consumes too much energy, ML could perform a partial computation to reduce the data to transmit and the frequency of communications.
tinyML presents endless possibilities, and the few mentioned are a small fraction of what is achievable. For those seeking to expand their knowledge and skills in this field, tinyML Foundation is the ideal community to join.
Join the tinyML community!
tinyML Foundation (www.tinyml.org) is a non-profit organization that aims to educate, inspire, and connect the worldwide tinyML community.
Supported by companies such as Arm, Edge Impulse, Google, and Qualcomm, the foundation is energizing a diverse global community of engineers, scientists, academics, and business professionals to envision a world of ubiquitous devices powered by tinyML to create a healthier and sustainable environment.
Through free virtual and in-person initiatives, the tinyML Foundation promotes knowledge sharing, engagement, and connection among experts and newcomers. In 2023, over 13,000 people joined the group, and there have been 47 Meetup groups in 39 countries.
With several Meetup (https://www.meetup.com) groups in different countries, you can join any near you for free (https://www.meetup.com/en-AU/pro/TinyML/) to always be up to date with new tinyML technologies and upcoming events.
After this brief introduction to tinyML, it is time to explore its ingredients in more detail. The following section will start analyzing the element that makes our devices capable of intelligent decisions.
