Arm Newsroom Blog

Transforming AI Experiences at the Edge with a System-Level Approach

Arm CPUs empower developers to drive new AI innovations
By Arm Editorial Team

The rapid evolution of AI on edge devices like laptops isn’t just transforming everyday life; it’s changing the way we think about how developers can exploit rapidly changing models to continue delivering amazing, world-changing experiences. 

Developers want to land their AI-based innovations quickly at the highest possible performance. This requires a range of heterogenous capabilities delivered in a compute platform.

The criticality of CPUs

The CPU is highly accessible and easy to program, making it ideal for developing applications. And continuous investments in the Arm architecture—the foundation of the Arm CPU—have led to the development of critical features for accelerating AI computation. These efforts include dot product instructions, support for the BFloat16 format, matrix multiply instructions for Int8 and BFloat16, and the powerful SVE2 vector extension. 

These advancements extend the capabilities of the latest Armv9 Cortex CPUs for applications requiring more compute performance for data processing and advanced workloads. This relentless focus on fast-paced architectural evolution prepares the developer ecosystem for ever-changing compute demands in the age of AI.

GPUs accelerate a broad array of parallelizable computations and for developers the AI benefits are accessed via APIs. Both the CPU and GPU can be seamlessly integrated with neural processing units (NPUs) in a heterogeneous system on chip (SoC) environment.

NPUs are tailored to efficiently process the operations commonly used in neural networks, such as matrix multiplications and convolutions. This specialization as an offload engine allows them to perform these tasks quickly to deliver enhanced AI performance. 

However, compared to CPUs and GPUs, NPUs are complex to program and integrate into SoCs, and have limited flexibility since they are designed for specific AI tasks. 

Enabling the growing developer ecosystem

One area benefitting from AI innovation is the current PC market, which is witnessing a rapid expansion of AI-based applications and features, enhancing a range of productivity and creativity tasks. AI-powered smart assistants, generative AI, and advanced AI workloads are transforming the imaging pipeline, a fundamental feature for PC video conferencing. This evolution is laying the foundation for the future growth of AI PCs.

Meanwhile, AI workloads will run more efficiently and faster within the memory bandwidth of PC designs via more mature and smaller LLMs, like Meta’s Llama3 and Microsoft’s Phi-3. Arm is already demonstrating how these new models can run entirely at the edge on the Arm CPU, with the impressive performance enabled by existing CPU instructions for AI and dedicated software optimizations. 

Supporting Arm-based PCs’ strong hardware and software capabilities is a growing developer ecosystem. Thanks to the expanded end-to-end Windows Arm-native developer toolchain, more Windows developers are creating Arm-native builds of their applications. This is being driven by vendors of leading applications, including Adobe Photoshop, Dropbox, Firefox, Microsoft Edge, Office, Teams, Visual Studio, Netflix, Spotify, WhatsApp, and Zoom, all providing Arm-native builds. These investments translate into tangible application improvements, such as improved startup times and smoother performance.

Arm CPU at the forefront of AI 

As the AI landscape continues to evolve, the importance of computing efficiency and design flexibility will only grow. The Arm CPU, with its continued advancements in AI capabilities, strong ecosystem support, and inherent advantages in latency and memory locality, is well-positioned to remain at the forefront of AI enablement in consumer devices. 

By adopting a system-level approach that leverages the strengths of the CPU alongside other specialized processors, the industry can continue to push the boundaries of AI innovation, while delivering faster, more responsive and more secure AI experiences to users at the edge.

The rise of AI-enabled edge devices and PCs represents an exciting new era for the market, where really the only gating factor to transformative innovation is imagination. Arm is continuously evolving technologies that enable AI wherever computing happens. 

For more information, please see our AI solutions page.

Article Text
Copy Text

Any re-use permitted for informational and non-commercial or personal use only.

Editorial Contact

Brian Fuller & Jack Melling
Subscribe to Blogs and Podcasts
Get the latest blogs & podcasts direct from Arm

Latest on Twitter