TinyML in 2024: Machine Learning at the Edge

From manufacturing to retail, applications of edge analytics transforming industries and the edge computing market is expected to reach ~$61B by 2028 with a compound annual growth rate of 38.4%. However, the current approach to edge analytics involves machine learning models trained on the cloud. This introduces latency to the system and is prone to privacy issues.

TinyML is a new approach to edge computing that explores machine learning models to be deployed and trained on edge devices.

What is TinyML?

Tiny Machine Learning (TinyML) is a field of study at the intersection of machine learning (ML) and embedded systems that enables running ML models on devices with extremely low-power microcontrollers. Let’s explain some terms.

Embedded systems are hardware and software systems designed to perform a dedicated function. They are computers, but in contrast to general-purpose computers such as a pc, a smartphone, or a tablet, embedded systems aim to perform specific tasks. Electronic calculators, digital cameras, printers, home appliances, ATMs are all examples of embedded systems.

Microcontrollers constitute the hardware part of an embedded system. These are chips consisting of a processor, RAM, ROM, and Input/Output (I/O) ports, enabling embedded systems to perform their task.

What are the features of microcontrollers?

Microcontrollers are:

• Low-power devices. A typical microcontroller requires power in the milliwatt or microwatt range, so they consume power more than a thousand times less than a standard computer.
• Cheap. More than 28 billion units of microcontrollers shipped in 2020.
• Prevalent. As you can see from the examples of embedded systems above, you are already using microcontrollers every day.

TinyML brings machine learning to microcontrollers and Internet of Things (IoT) devices to perform on-device analytics by leveraging massive amounts of data collected by them.

Why is TinyML important now?

TinyML delivers intelligence to low-memory and low-power tiny devices by enabling machine learning on them. 

A standard IoT device collects data and sends it to a central server over the cloud where the hosted machine learning models provide insights. TinyML optimizes ML models to work on resource-constrained edge devices. It eliminates the necessity of data transmission to a central server and opens up new possibilities by bringing intelligence to millions of devices that we use every day. The graph below reflects the level of interest in TinyML.

What are the advantages of TinyML?

• Fast inference with low latency: Since TinyML enables on-device analytics without the necessity of sending data to a server, edge devices can process data and provide inference with low latency.
• Data privacy: Keeping the data on the edge device reduces the risk of sensitive data being compromised.
• Doesn’t depend on connectivity: With TinyML, smart edge devices can make inferences without an internet connection.

What are the challenges facing TinyML?

• Limited memory: TinyML devices have kilobytes or megabytes of memory. This puts restrictions on the size and the runtime of the machine learning models deployed on these devices. Currently, there is a limited number of ML frameworks which can meet the requirements of TinyML devices. TensorFlow Lite is one such framework.
• Troubleshooting: Since the ML model trains on the data that the device collects and runs on the device itself, it is harder to determine and fix the performance issues than in a cloud setting where troubleshooting can be done remotely.

What are the use cases and applications of TinyML?

TinyML has the potential to change the settings where IoT data is utilized with reduced latency and improved privacy. Industries that can benefit from TinyML include:

• Manufacturing: TinyML-powered predictive maintenance can reduce the downtime and costs associated with equipment failure.
• Retail: TinyML can be used to monitor inventories and send alerts. This can prevent out-of-stock situations.
• Agriculture: TinyML devices can be used to get real-time data by monitoring crops or livestock.
• Healthcare: Real-time health monitoring enabled by TinyML devices can deliver better and more personalized patient care.

An example of a real-world application of TinyML in healthcare is hearing aids. Hearing aid hardware is battery powered and runs on resource-constrained microcontroller units which limit the size of neural networks required to achieve satisfactory performance. Researchers applied model compression techniques and achieved lower latency without a statistical difference in listening preference.

How to implement TinyML?

There are a couple of machine learning frameworks that support TinyML applications. These are:

• TensorFlow Lite
• PyTorch Mobile
• Edge Impulse

If you want to read more on analytics and computing on edge devices, check our articles:

• Edge Analytics: What it is, Why it matters & Use Cases
• IoT Analytics: Benefits, Challenges, Use Cases & Vendors

If you have other questions about TinyML, feel free to contact us: