Arm to Supercharge AI on Android, Powering Next-Gen Features Across Mobile Apps

From sharpening a photo just before the shutter clicks to real-time background noise suppression during a call, or even interacting with an AI assistant offline, on-device AI is redefining mobile experiences. Powering these seamless interactions is the Arm compute platform.

But as AI workloads become more sophisticated and pervasive, mobile developers face increasing demands to deliver real-time, efficient performance within strict and diverse constraints on memory, power, and thermal budgets, without adding further complexity to their apps and workloads.

To meet these challenges, Arm introduced Scalable Matrix Extension 2 (SME2) to accelerate matrix heavy workloads – that are essential for computer vision and generative AI applications – directly on the mobile CPU. SME2 is a set of advanced CPU instructions in the Armv9 architecture built on the foundation of SME that enable real-time mobile inference tasks, such as image and natural language processing and voice generation, as part of a heterogeneous computing approach to AI.

How developers access SME2 benefits

SME2 benefits are readily accessible to mobile developers with no changes required to their existing code, models or applications. This is made possible through Arm KleidiAI, Arm’s software acceleration layer which seamlessly integrates in leading runtime libraries and AI frameworks.

Thanks to deep and extensive KleidiAI integrations, SME2 is enabled in Google’s XNNPACK, a highly optimized neural inference library for Android, and across multiple frameworks, including Alibaba’s MNN, Google’s LiteRT and MediaPipe, Microsoft’s ONNX Runtime, and llama.cpp. These integrations mean that SME2 is already embedded within the software stack. When SME2 is enabled and compatible, XNNPACK automatically routes the matrix heavy operations to SME2 via KleidiAI, so developers directly benefit with no changes needed in application logic or infrastructure.

At the same time, KleidiAI is not just about today’s acceleration – it is architected for the future. As Arm introduces new features and architectural advancements, KleidiAI will continue delivering automatic performance improvements to supported AI frameworks and runtime libraries without requiring additional developer effort.

AI performance and efficiency optimizations at scale

For mobile developers, the performance benefits from SME2 are coming soon on Android in upcoming smartphones with SME2-enhanced hardware, and already available in the latest iOS devices – with the full device list here – provided their apps use the supported AI frameworks.

When executed on SME2-enhanced hardware, Google’s Gemma 3 model delivers 6x faster AI responses when running on chat interactions compared to the same device without SME2 active. Also, on a single CPU core with SME2 acceleration, Gemma 3 can start text summarization of up to 800 words in under one second. Meanwhile, a leading ISV has already committed to moving the majority of its application token generation from the cloud to mobile thanks, in part, to advancing on-device AI capabilities.

“SME2-enhanced hardware enables more advanced AI models, like Gemma 3, to run directly on a wide range of devices. As SME2 continues to scale, it will enable mobile developers to seamlessly deploy the next generation of AI features across ecosystems. This will ultimately benefit end-users with low-latency experiences that are widely available on their smartphones.” Iliyan Malchev, Distinguished Software Engineer, Android at Google.

Alongside performance enhancements, SME2 enables portable solutions across Arm-based ecosystems, like iOS and Android, that feature nine million apps running on Arm and more than 22 million software developers building their apps and workloads on Arm.

Enabling developers to build the future of AI

Our message to Android developers is simple: make sure your apps are built on top of AI frameworks and runtime libraries with KleidiAI integrations now, so you can automatically benefit from SME2 and ship next-gen AI features across devices.

By preparing today, you ensure your apps are optimized for SME2-enhanced hardware as it rolls out, unlocking faster performance, lower latency, and better efficiency for your users without rewriting a single line of code.

More information about how to plan for SME2 can be found in this Arm Community blog. We have also released a developer launchpad to help mobile developers utilize the benefits of SME2, with extensive examples showing how to target KleidiAI enhanced frameworks today to pick up the latest accelerations by default in the hardware.