How Arm is helping developers build local AI systems for the intelligent edge 

The next phase of AI is being shaped by interaction, context, and immediacy. 

A robot identifying a speaker, a smart camera interpreting its environment, or an accessibility device responding to a user are examples. These systems increasingly combine vision, speech, language, sensor data, and movement, then turn those inputs into action in real time.  

They are also becoming more agentic. Rather than execute a single model or fixed inference task, applications must reason across inputs, select models or tools, maintain context, interact with devices, and trigger actions.  

This makes edge AI much more than a model deployment challenge. Developers must build complete intelligent systems spanning CPUs, GPUs, accelerators, sensors, AI runtimes, applications and cloud services. And they must decide where everything should run. 

Latency affects whether an interaction feels immediate. Cameras and microphones create privacy considerations. Devices operate within power and thermal limits. Networks may be intermittent or unavailable.  

The result is not a choice between cloud and edge, but a distributed computing continuum where developers decide what should run locally, what should run in the cloud, and how those parts work together in a complete agentic system. 

Developers are already building the local AI loop 

We can already see this shift in what edge developers are building on Arm.  

At the Arm Create Builder Zone at We Are Developers, attendees profiled workloads running on Arm, explored on-device generative AI, and connected AI applications to Arm-powered devices. 

One live translation application at the Arm Create events showed this complexity, using more than 30 models across multiple stages to create a single responsive experience. Different parts of the pipeline used different frameworks and numerical precisions, depending on whether each stage was compute-bound or memory-bound. 

In another example, Qwen3-TTS CustomVoice ran in real time on a single Arm CPU core using Arm Scalable Matrix Extension 2 (SME2), demonstrating how advanced speech generation workloads can now run locally without requiring a dedicated cloud round trip. 

Finally, the Reachy Mini and NVIDIA DGX Spark demonstration shows how a physical AI system can bring perception, conversation, and movement into one local loop. Instead of treating vision, language, and control as isolated workloads, developers must coordinate them as parts of one responsive system. 

Agentic patterns matter here. A robot must interpret context, select an action, invoke a device capability, and continue adapting as its environment changes. 

Across these examples, the development challenge is similar: coordinating increasingly complex AI workflows while preserving the responsiveness, privacy, and efficiency benefits of local processing. 

Optimizing the system, not just the model

Building these experiences also means making decisions across the complete application. 

Which model should perform each task? Which runtime should it use? Should it run on the CPU, GPU or another accelerator? What numerical precision provides the right balance between performance and quality? And should a particular action happen locally or use the cloud? 

No single answer fits the entire application. 

The Arm Create translation example combined multiple edge AI frameworks, including ExecuTorch, LiteRT, llama.cpp and ONNX Runtime. For quantization, some workloads require FP32 or FP16, while others can run efficiently using INT8, INT4 or lower precision. 

The goal isn’t to force everything onto one model, framework or compute engine. It’s to match each workload to the approach that makes most sense for the complete experience. 

Agentic AI makes that flexibility even more important because these decisions can increasingly happen dynamically as applications reason, select tools and determine what to do next. 

Making complex AI development easier 

As applications become more sophisticated, developers need a simpler way to navigate the complexity and choice available.  

Arm AI Portal brings resources into a connected workflow for developers and coding agents. Developers can discover pre-optimized models, evaluate performance and accuracy for specific Arm-based targets, and assess trade-offs such as model size, target platforms, quality, and expected performance. The portal also provides access to code, tools, and deployment workflows that shorten the path from model selection to implementation.    

But AI isn’t only changing what developers build. It’s changing how they build it. 

Through the Arm Model Context Protocol (MCP) Server, coding agents can access relevant Arm resources directly within developer workflows, helping connect model discovery, evaluation, optimization and deployment. 

Together, AI Portal and MCP-enabled development reflect the same trend as the applications themselves: as systems become more connected, so must the tools used to build them. 

Edge AI is becoming multimodal, agentic, and physical 

As more intelligence runs across devices, gateways, robots, cameras, vehicles, and local compute systems, applications will combine more modalities and become increasingly capable of acting. 

As that happens, orchestration becomes a core edge AI requirement. Applications must connect reasoning with models, local data, tools, device capabilities, and cloud services without sacrificing the constraints that made local processing necessary in the first place. 

The range of local workloads is expanding, from transcription, dictation, and image enhancement to on-device assistants, expressive speech generation, translation, and other multimodal experiences. 

Governance also becomes part of the system design. As applications gain more autonomy, teams need to consider how capabilities are exposed, what actions agents are permitted to take, and where control should remain local. 

Developers need consistent paths across heterogeneous systems, with the flexibility to select models, frameworks, precision, and compute resources for each workload across cloud, edge, and physical devices. Workflows must also make those resources accessible to people and coding agents. 

That is the focus of the next Arm Create at PyTorch. At the event, developers can explore PyTorch and ExecuTorch workflows, agentic AI and MCP, and edge and physical AI through technical sessions, live demos, and hands-on experiences. 

Turning intelligence into action 

The value of local AI is clearest in the experiences it enables. Robots can interpret their environments and act with less dependence on a cloud connection. Accessibility applications can process sensitive information close to the user. Speech and translation systems can coordinate multiple specialized models while remaining responsive. Smart cameras and industrial systems can turn local sensor data into timely action. 

Across these use cases, the developer challenge is shifting from deploying individual models to designing complete intelligent systems. That’s the opportunity Arm is building for: giving developers and their coding agents consistent ways to discover, connect, test and optimize the models, software and compute capabilities behind those systems. 

As AI becomes more multimodal and agentic, the edge won’t simply be another place to run inference. It will increasingly be where intelligence understands context, makes decisions and turns them into action.