Arm: The Technology Foundation for AI Everywhere

What makes Arm the foundation for AI innovation?

Companies using Arm’s common and ubiquitous computing platform have shipped 270 billion chips to date, and today, Arm enables AI capabilities across all technology touchpoints. It is a key reason why AI leaders are able to innovate at such an incredible pace on Arm.

Why is Arm essential for running AI?

Arm is and will be foundational to everything happening with AI. Put simply, AI runs on Arm now, and will in the future.

How is Arm meeting the demand for AI everywhere?

Meeting the insatiable demand for AI-based technologies and applications requires AI everywhere across every corner of computing.

What is Arm’s role across the computing spectrum?

These vary from the big data centers that use countless processors to train large language models (LLMs) right through to AI at “the edge” – on the actual devices – that are used every day by the world’s digital consumers.

Here’s how Arm is achieving its mission to enable AI everywhere.

How is Arm contributing to efficient AI computing?

The high compute demands from AI need to be matched by a commitment to efficient computing.

• Unsustainable computing resources – Advancing Generative AI is driving the need for an unprecedented amount of AI computing power and resources. One Dutch researcher has suggested that if every Google search for a year used AI, it would require the equivalent amount of electricity used to power a country the size of Ireland. This is not sustainable in the long-term, especially with finite resources on our planet.
• The smartphone parallel – Arm delivers a unique combination of computing performance and power efficiency that fueled the smartphone revolution, and we see the same happening as part of the AI revolution.

How is Arm transforming AI in the data center?

Hyperscale leaders are embracing custom silicon strategies to transform their data centers for the era of AI.

• Grace Hopper Superchip – SoftBank is building the world’s first 5G AI data center with NVIDIA’s Arm-based Grace Hopper Superchip, a breakthrough AI-accelerated CPU designed for giant-scale AI and high-performance computing applications. Powered by 72 Arm Neoverse V2 cores, Grace Hopper provides a 10x performance leap in AI’s most demanding tasks, while dramatically improving performance per watt.
• Cobalt 100 – Microsoft recently announced its first custom silicon built for the cloud, the Cobalt 100 processor, which is built on the Arm Neoverse CSS N2.
• Significant costs savings – The Arm Neoverse-powered AWS Graviton3 is yielding 50 percent cost savings for AI inference – the process of using a trained model to make predictions on new real-world data.
• Important to focus on AI inference – As around 85 percent of AI workloads in data centers run inference – which powers AI applications across countless use cases, industries and devices – there is a massive incentive for companies to lower the associated energy and costs. 

How is Arm enabling AI at the edge?

Alongside the data center and cloud, many AI workloads will need to be processed on the actual devices to distribute AI more effectively.

• Edge AI is happening now – The processing of AI workloads on the actual devices – whether that’s a smartphone, smart home device or even the car – is already a reality today and will continue to grow, especially as LLMs become more efficient and purpose-built for specific tasks, use cases or devices.
• Faster experiences, less delays, improved privacy – Processing AI compute on the device, where the data is captured, means quicker, more secure user experiences and less delays – a significant requirement for safety-critical use cases like industrial and automotive applications.
• AI coding for small processors – Developers are writing more compact AI models to run on small microprocessors or even smaller microcontrollers, which saves energy, time, and costs. For example, Plumerai, which provides software solutions for accelerating neural networks on Cortex-A and Cortex-M-based SoCs, runs just over 1MB of AI code on an Arm-based microcontroller that performs facial detection and recognition.

How is AI advancing processor technologies across diverse technology markets?

AI starts on the CPU, whether it is handling AI workloads in their entirety or being supported by co-processors like GPUs or NPUs.

• Advancing AI capabilities – Arm’s CPU and GPUs have doubled their AI processing capabilities every two years during the past decade, leading to 70 percent of AI in today’s third-party applications running on Arm CPUs. This translates to significant AI developments at the edge across key technology markets.
• AI-powered smartphones are here now – High performance AI-enabled smartphones are now on the market, and these are built on Arm’s v9 CPU and GPU technologies. These include the new MediaTek Dimensity 9300-powered vivo X100 and X100 Pro smartphones, and the Google Pixel 8, which is the first smartphone engineered for Gemini Nano – Google’s AI model built for a range of AI-based tasks at the edge.
• Running machine learning workloads for IoT – Over 20 billion Arm-based systems-on-chip (SoCs) are capable of running a variety of machine learning (ML) workloads for a broad spectrum of IoT devices, ranging from those built on Cortex-M processors to Ethos-U NPUs to Cortex-A CPUs.
• Driving AI in Automotive – Today’s chips in cars enable fundamental AI-enabled capabilities, like obstacle detection, 3D views and simple sensor fusion, which all run on the Arm CPU. The next-generation of AI-based automotive features, like advanced driver-assistance systems (ADAS) and autonomous driving capabilities, are also being built on Arm CPUs.

How is Arm cultivating the largest AI developer community in the world and accelerating AI workloads?

Arm’s continuous software investments mean we are creating the largest AI developer community globally.

• A commitment to software developers – For developers, Arm aims to enable AI everywhere in the most common way possible for easier, faster, more secure coding. Today, over 100 million users have access to the Arm NN software development kit to optimize their ML workloads on Arm CPUs and GPUs.
• More complex AI workloads, quicker time-to-market – For the 15 million developers worldwide developing for Arm-based devices, this allows them to run complex AI and ML workloads and ensure they get their applications to market faster.
• Partnerships with industry leaders – Arm is working across our industry-leading ecosystem on a range of partnerships to make our AI commitments to developers a reality.
• Open-source support – We are also enabling Arm-based hardware with increased ML capabilities through open-source frameworks and libraries in all the places where developers need support, including TensorFlow, PyTorch, Caffe 2, OpenVINO and TVM – creating an AI innovation foundation for the open-source community.

Some final thoughts

The foundation for technology innovation – AI is providing the foundation for the next era of technology innovation, ushering in a new era of possibilities for society.

But we need AI everywhere to make this a reality – This not only means enabling AI in the cloud, data center and supercomputer, but making sure complex AI workloads can take place on the edge across far smaller, more constrained technologies and devices.

Power efficiency equal to performance – To grow edge AI, power efficiency needs to have equal prominence to performance.

Arm is making this happen – We provide a common and ubiquitous compute platform that enables these possibilities at the edge and beyond.

• Architecture and processors – Whether it’s our CPUs, GPUs, ISPs or NPUs, Arm adds more AI performance and efficiency capabilities and security features to each generation, alongside unmatched flexibility for partners to integrate and develop their own innovative solutions.
• Software, tools and ecosystem – IP combined with open software and tools and a vast industry-leading ecosystem means that millions of developers worldwide are using the Arm compute platform as the foundation for their AI-based innovation.

AI on Arm is everywhere – From sensors and smartphones to industrial-scale IoT, cars and data centers.