TinyML – Everything You Need to Know

Machine Learning models are all over the place right now. During the day, you probably utilize these models a lot more than you realize. Machine learning models are utilized in common tasks such as social media browsing, photographing, and checking the weather.

A machine-learning algorithm may have recommended this blog to you. We’ve all heard about how time-consuming it is to train these models. We’ve all heard that training these models is time-consuming. 

However, doing inference on these models is frequently computationally costly.

We need computer systems that are fast enough to handle the rate at which we are utilizing machine learning services. As a result, the majority of these models are run on massive data centers with CPU and GPU clusters (even TPUs in some cases). 

When you take a picture, you want machine learning to instantly improve it. You don’t want to have to wait for the image to be transferred to a data center, processed, and returned to you. In this case, the machine learning model should be executed locally. 

When you say “Hey Siri” or “OK, Google,” you want your gadgets to respond immediately. Waiting for your voice to be transmitted to the computers, where it will be evaluated and data obtained. 

This takes time and has a detrimental effect on the user experience. In this case, you want the machine learning model to work locally as well. This is where TinyML comes in. 

In this post, we will look into TinyML, how it works, its uses, how to get started with it, and much more.

What is TinyML?

TinyML is a cutting-edge discipline that applies the revolutionary potential of machine learning to the performance and power limits of small devices and embedded systems. 

Successful deployment in this industry necessitates a thorough understanding of applications, algorithms, hardware, and software. It is a machine learning subgenre that uses deep learning and machine learning models in embedded systems that employ microcontrollers, digital signal processors, or other ultra-low-power specialized processors. 

TinyML-enabled embedded devices are intended to run a machine learning algorithm for a specific job, typically as part of the device’s edge computing. 

In order to run for weeks, months, or even years without recharging or battery replacement, these embedded systems must have a power consumption of less than 1 mW.

How does it work?

The only machine learning framework that can be utilized with microcontrollers and computers is TensorFlow Lite. It’s a set of tools that let developers run their models on mobile, embedded, and edge devices, allowing for machine learning on the fly. 

The interface of the microcontroller is utilized to collect data from sensors (like microphones, cameras, or embedded sensors).

Before being sent to the microcontroller, the data is incorporated into a cloud-based machine learning model. Batch training in offline mode is commonly employed to train these models. The sensor data that will be used for learning and inference has already been determined for the specific application. 

If the model is being trained to detect a wake word, for example, it is already set up to handle a continuous audio stream from a microphone. 

Everything is already done with the aid of a cloud platform like Google Colab in the case of TensorFlow Lite, including dataset selection, normalization, underfitting or overfitting of the model, regularization, data augmentation, training, validation, and testing. 

A fully trained model is eventually transformed and transferred to the microcontroller, microcomputer, or digital signal processor after offline batch training. The model has no additional training after being moved to an embedded device. Instead, it solely uses real-time data from sensors or input devices to apply the model. 

As a result, a TinyML machine learning model must be exceptionally durable and capable of being retrained after years or never being retrained. All potential model underfitting and overfitting must be investigated so that the model remains relevant for an extended length of time, ideally indefinitely.

But Why use TinyML?

TinyML began as an endeavor to eliminate or lessen the IoT’s reliance on cloud services for basic small-scale machine learning operations. This necessitated the use of machine learning models on the edge devices themselves. It provides the following major benefits:

1. Low-power consumption: A TinyML application should preferably utilize less than 1 milliWatt of power. With such low-power consumption, a device may continue to derive conclusions from sensor data for months or years, even if powered by a coin battery.
2. Lower cost: It is designed to run on low-cost 32-bit microcontrollers or DSPs. These microcontrollers are typically a few cents each, and the total embedded system developed with them is less than $50. This is a very cost-effective option for running little machine learning programs on a big scale, and it is especially beneficial in IoT applications where machine learning must be applied.
3. Lower Latency: Its applications have low latency since they don’t need to transport or exchange data over the network. All sensor data is recorded locally, and conclusions are drawn using a model that has already been trained. The results of inferences may be sent to a server or cloud for logging or additional processing, although this is not essential for the device to work. This minimizes network latency and eliminates the need for machine learning operations to be performed on a cloud or server.
4. Privacy: It is a major concern on the internet and with the internet of things. The machine learning work in TinyML apps is performed locally, without storing or sending sensor/user data to a server/cloud. As a result, even while linked to a network, these applications are safe to use and pose no privacy risks.

Applications

1. Agriculture – When farmers take a photo of a plant, TensorFlow Lite’s application detects illnesses in it. It works on any device and does not require an internet connection. The procedure protects agricultural interests and is a critical necessity for rural farmers.
2. Mechanics Maintenance  – TinyML, when used on low-powered devices, can continually identify flaws in a machine. It entails prediction-based maintenance. Ping Services, an Australian start-up, has introduced an IoT gadget that monitors wind turbines by attaching itself to the turbine’s outside. It notifies the authorities whenever it detects any possible problem or malfunction.
3. Hospitals – The Solar Scare is a project. Mosquito use TinyML to halt the spread of illnesses such as dengue and malaria. It is powered by solar energy and detects mosquito breeding conditions before signaling the water to inhibit mosquito breeding.
4. Traffic Surveillance – By applying TinyML to sensors that collect real-time traffic data, we can use them to better direct traffic and cut response times for emergency vehicles. Swim.AI, for example, uses this tech on streaming data to increase passenger safety while also reducing congestion and emissions through smart routing.
5. Law: TinyML can be used in law enforcement to identify illegal actions such as rioting and theft using machine learning and gesture recognition. A similar program may also be used to secure bank ATMs. By watching user behavior, a TinyML model may predict whether the user is a real consumer completing a transaction or an intruder attempting to hack or destroy the ATM.

How to get started with TinyML?

To get started with TinyML in TensorFlow Lite, you’ll need a compatible microcontroller board. TensorFlow Lite for Microcontrollers supports the microcontrollers listed below.

• Wio Terminal: ATSAMD51
• Himax WE-I Plus EVB Endpoint AI Development Board
• STM32F746 Discovery kit
• Adafruit EdgeBadge
• Synopsys DesignWare ARC EM Software Development Platform
• Sony Spresense
• Arduino Nano 33 BLE Sense
• SparkFun Edge
• Adafruit TensorFlow Lite for Microcontrollers Kit
• Adafruit Circuit Playground Bluefruit
• Espressif ESP32-DevKitC
• Espressif ESP-EYE

These are 32-bit microcontrollers having enough flash memory, RAM, and clock frequency to execute a machine learning model. The boards also have a number of onboard sensors capable of running any embedded program and applying machine learning models to the targeted application. To build a machine learning model, you will need a laptop or computer in addition to a hardware platform.

Each hardware platform has its own programming tools for building, training, and porting machine learning models, which use the TensorFlow Lite for Microcontrollers package. TensorFlow Lite is free to use and modify because it is open source.

To get started with TinyML and TensorFlow Lite, all you need is one of the above-mentioned embedded hardware platforms, a computer/laptop, a USB cable, a USB-to-Serial converter – and the desire to practice machine learning with embedded systems.

Challenges

Even while TinyML’s progress has yielded many positive results, the machine learning industry still confronts considerable obstacles.

• Software diversity – Hand-coding, code generation, and ML interpreters are all options for deploying models on TinyML devices, and each takes a different amount of time and effort. Different performances can arise as a result of this.
• Hardware diversity – There are several hardware options available. TinyML platforms can be anything from general-purpose microcontrollers to cutting-edge neural processors. This causes issues with model deployment across different architectures.
• Troubleshooting/debugging – When an ML model performs poorly on the cloud, it’s simple to look at the data and figure out what’s going wrong. When a model is spread across thousands of TinyML devices, with no data stream returning to the cloud, debugging becomes difficult and may necessitate a different method.
• Memory constraints – Traditional platforms, such as smartphones and laptops, need gigabytes of RAM, whereas TinyML devices use kilobytes or megabytes. As a result, the size of the model that may be deployed is limited.
• Model training – Although there are several advantages to deploying ML models on TinyML devices, the bulk of ML models is still trained on the cloud to iterate and continuously improve model accuracy.

Future

TinyML, with its small footprint, low battery consumption, and lack of or limited reliance on internet connectivity, has enormous potential in the future, as the majority of narrow artificial intelligence will be implemented on edge devices or independent embedded gadgets. 

It will make IoT applications more private and secure by leveraging them. Though TensorFlow Lite is currently the sole machine learning framework for microcontrollers and microcomputers, other comparable frameworks such as sensor and ARM’s CMSIS-NN are in the works. 

While TensorFlow Lite is an open-source project in progress that got off to a terrific start with Google’s Team, it still needs community support to get into the mainstream.

Conclusion

TinyML is a novel approach that combines embedded systems with machine learning. As the narrow AI peaks in many verticals and domains, the technology can emerge as a prominent subfield in machine learning and artificial intelligence. 

It provides a solution to numerous challenges that the IoT sector and professionals applying machine learning to many domain-specific disciplines are now facing. 

The concept of utilizing machine learning at edge devices with a small computing footprint and power consumption has the potential to significantly transform how embedded systems and robotics are constructed.