How on-device AI acceleration effortlessly improves everyday apps

Let’s say you’re texting someone, and the phone translates what you’re typing mid-sentence as well as summarizes the entire conversation. Or you’re taking a photo and it instantly interprets a busy scene and highlights what matters before you hear the shutter sound. Mobile applications that once paused to process now anticipate what you need in real-time.

This is a direct response to a world where every tap, swipe and second of attention counts. Today’s users expect their apps to think fast as they do, respond instantly, and protect their private information.  These demands filter through to developers like you who are under increasing pressure to do more than just make apps run, which means you need to deliver advanced real-time experiences entirely on the device without relying on the cloud, burning battery, or having response lags.

Fortunately for the millions of mobile developers worldwide, Arm Scalable Matrix Extension 2 (SME2), provides the on-device AI acceleration capabilities that help make faster, smarter experiences a reality.

Acceleration that just works, everywhere

Enabled automatically through Arm KleidiAI integrations into the most widely-used frameworks, including Google LiteRT, PyTorch ExecuTorch, ONNX Runtime, and Alibaba’s MNN, SME2 moves AI computation onto the Arm CPU – the compute platform that powers around 99% of smartphones today. This gives developers greater flexibility with their workloads, while freeing up bandwidth on the GPU.

Since the Arm CPU can handle mixed compute pipelines on a secure, consistent architecture, you don’t have to rebuild your approach for every device tier. Just optimize once and your work carries across the ecosystem. Through extending the Armv9 architecture with dedicated matrix-processing instructions, SME2 enables CPUs to handle the same dense math operations that power today’s large language and vision models, but with far lower power and latency.

The result is more engaging, intelligent, intuitive applications that run across billions of mobile devices, backed by real, measurable gains:

• Up to 5x faster AI responses
• 4.7x lower latency for speech workloads
• 2.8x faster audio generation
• Notable power savings in common mobile AI scenarios.

That’s how Alipay can now generate travel videos instantly on the device, or how Google delivers real-time summarization and translation through their range of apps.

For users, this means app experiences that are fast, local, and trustworthy. For developers, it expands what’s possible within the same thermal and power budgets, with real-time inference that no longer drains the battery or overheats the phone. Developers can also experiment freely, scale across hardware generations, and deliver the kind of AI performance once reserved for servers – all on the CPU.

Build once, run everywhere

Every developer knows the challenge of fragmentation, which includes different chipsets, different performance profiles, endless rounds of tuning. SME2 standardizes AI acceleration through the CPU, giving developers a single, portable foundation for mobile AI.

Apps built on SME2 perform consistently across iOS and Android, from flagship phones such as the vivo X300 to iPhones 16 and 17, to mid-to-entry tier smartphones powered by Arm CPUs. Whether you’re optimizing a vision model for a creative app or deploying a voice assistant for global markets, your code performs predictably. Testing becomes simpler, performance regressions rare, and time-to-market faster.

As Arm evolves its architecture, KleidiAI continues to deliver performance improvements automatically, with no coding reworks needed. That’s the value of a platform that grows with you. Update your framework, flip SME2 on, and see how much faster your app can go.

The next frontier of on-device intelligence

The takeaway here is that on-device AI is less about adding more intelligence to the device, but more about bringing that intelligence closer to the user and their day-to-day app experiences.

At the same time, users won’t wait for technology to catch up, and neither should you. SME2 through KleidiAI gives you the power to close that gap, turning the CPU into a ready-made AI engine that keeps pace with every tap and interaction.

So if you’re building the next great mobile experience, start where it matters most: at the core. Update your frameworks, enable SME2, and see what your app can do when intelligence runs at the speed of your users.