As part of our Talking AI campaign, we’re exploring AI across Sheffield’s digital ecosystem. In this post, guest author Pierre Drezet, experienced systems engineer and founder of inx, shares his perspective on Edge AI. Pierre explains why running AI on-device – rather than in the cloud – is creating new opportunities and real-world benefits for businesses.

Edge AI – not such a new idea

If you use Alexa, Siri, or a smartphone camera, you already use edge AI. Features like wake-words and photo processing use the same “neural computing” techniques found in chatty, generative AI, but they run locally on your device.

Edge AI means performing AI computation on a local device at the “edge” of the network, rather than in a central cloud data centre.

Adaptive systems have been part of our lives for many years. Long before the current focus on hyper-complex models, we depended on local AI for face recognition, noise suppression, and medical devices – intelligent systems don’t always require data centres or device connectivity.

Getting started with edge AI

Raspberry Pis and Arduinos are common starting points for experimenting with edge AI.The next step is typically to build or download models using frameworks like TensorFlow-Lite/LiteRT (Google), ExecuTorch (Meta), TensorRT (Nvidia) and the ONNX interoperable format, to generate optimised inference models specifically targeting edge hardware.

There are a range of open-source tools and SDKs from companies such as Google, Meta and Nvidia, alongside cloud-based platforms like Google Colab and Edge Impulse for training and optimising machine learning models. For common edge AI applications such as computer vision, these frameworks also provide access to  pre-trained models that make getting started more straightforward.

Other applications, particularly those involving time-series data, require greater engineering expertise and domain knowledge, as the data often needs significant preparation before it can be used effectively.

Perhaps the biggest difference between edge AI and the generative AI tools most people are familiar with is what happens after the model produces its output. Rather than generating content for a person to interpret, edge AI often needs to make or trigger real-world decisions and actions. That demands additional engineering to ensure systems are reliable, safe and fit for purpose, much like the agentic AI and function-calling capabilities now emerging in smaller language models.

The potential of edge AI

Edge AI is a major focus for the silicon industry, with Neural Processing Units (NPUs) common in chips ranging from $2 micro-controllers to $20k gateways. The market is projected to grow five-fold by 2031, driven by demand for genuinely smarter products, privacy, lower operating costs, cheaper user interfaces, reliability, low-latency and fully disconnected use-cases.

This ecosystem is becoming practical for all developers. MLOps tools help develop models for consumer electronics and medical devices, making edge AI accessible and affordable for businesses of all sizes while meeting strict regulatory requirements.

SMEs are now building intelligent wearables and mobility products that once required massive resources. Rapid silicon innovation is providing efficient, ready-to-use components. Smaller Language Models (SLMs) and vision models are bringing conversational and agentic interfaces to smarter devices to make them easier to use and more robust to their environments. So, you don’t need to be a data scientist to build many edge AI products, where optimised models are already easily accessible.

Benefits of edge AI

Developing specialised, application-specific systems is vital. Since AI reliability varies, good product management – rather than just general intelligence – is what makes these products successful.

Key benefits include:

• Low latency for closed-loop control, alarms and responsiveness.
• Privacy and data sovereignty — sensitive data can be processed and acted on with or without storing or transmitting any meta data derived from the environment.
• Operating cost and energy efficiency — connectivity is expensive at scale, but local inference or compression is free and has a lower energy requirement.
• Disconnected operation — remote, underground, or shielded locations where connectivity is not available or desired for product scope reasons.
• Purpose-specific – embodying AI in a device and reducing functional scope to only that which is needed is easier to validate and maintain with an embedded implementation. Purchasers and users trust products that do one thing well over those that can do many things.

Edge AI makes public spaces safer while protecting identities, as data is processed locally rather than being sent to remote providers. Your (or your neighbour’s) door bell company may be aware of your comings and goings and your energy provider may be aware when you take a bath but these intrusions are far more accountable and limited than providing constant raw data to remote service providers.

Challenges

The capability wars of the AI hyperscalers has largely followed a narrative that if AI becomes increasingly capable then it becomes more useful. For physical AI (at least) this misses some basic and important performance metrics associated with autonomous systems. Without a human-in-the-loop, edge AI systems eventually produce a decision and action of one type or another – and the harsh realities of being wrong come to light. So, we must ask:

• What known level of confidence does that inference have?
• What is the cost of a false positive or false negative for each action?
• What unknowns are contributing to your answer?

Ultimately, performance is measured by error rates and the cost to the user. These are not easy metrics to measure or understand, but they are fundamental specifications for designing edge AI systems, built on trust and reliability. In edge AI systems, a data set’s greatest value is its ability to validate the product, not just train the model.

Environmental and condition variables – like sensor calibration drift, aging components, user behaviour or lighting changes – can degrade a model’s inference over time. Over-the-air (OTA) maintenance is essential for security and reliability, offering a safety net for managing non-stationary environments and system drift. And because field performance must be monitored, most edge AI products are often also IoT systems.

Edge AI improves security because less data is transmitted. New regulations like the EU’s Cyber Resilience Act (CRA) now mandate security updates and validation for IoT and edge AI devices, ensuring higher standards for end-users, and important risk classes of products which require third party assessment.

Use Cases

Edge AI is likely to remain a highly fragmented market but defragmentation of the technologies used and vendor ecosystems will be vital for cross-fertilisation across sectors with different price-points, constraints and development life-cycles. In this list of use cases, the same underlying techniques are used across sectors – from monitoring industrial pumps to detecting arrhythmias in wearables:

• Manufacturing:
• QA systems, anomaly detection, productivity, metrology, safety.
• Utilities:
• Predictive maintenance, smart sensors, real-time control.
• Robotics:
• Scene perception, localisation, planning, motion control.
• Consumer products:
• User interfaces, energy management, home security, home automation.
• Automotive:
• Self driving vehicles, hazard detection, diagnostics, UX, infotainment.
• Medical:
• Radiology and histology image analysis, wearable health monitoring.
• Defence:
• Radar analysis, electromagnetic measures, autonomous systems.
• Agricultural/environmental
• Irrigation, disease detection, biodiversity, precision weed control.

Enterprise and edge AI are converging in many ways. Enterprise is moving towards more focused models where trust and autonomy matter more than general capability. Enterprise AI is already wrestling with reliability, cost and runaway model sizes, and has much to learn from edge practice and technology.