Scaling Agentic AI: Arm AGI CPU and Red Hat bring production-ready AI stack to empower agentic AI data centers

As AI shifts from training to inference—driven by the rise of agentic systems — datacenters are entering a new phase of growth. The challenge is no longer just raw performance, but how to efficiently scale and orchestrate always-on, distributed AI workloads across hybrid environments.

Agentic AI systems operate continuously, coordinating tasks, interacting with data pipelines, and making decisions in real time. This shift is placing new pressure on infrastructure, requiring higher efficiency, greater compute density, and seamless orchestration across the stack.

A full-stack approach to AI at scale

Arm is collaborating with Red Hat to address this challenge with a fully integrated, production-ready AI stack. By combining the Arm AGI CPU with Red Hat’s enterprise open source platforms, the collaboration delivers a consistent foundation for running AI agents, cloud-native workloads, and enterprise applications across both cloud and on-prem datacenters.

At the software layer, Red Hat provides a complete enterprise-grade platform optimized for Arm architecture. Red Hat Enterprise Linux (RHEL) on Arm delivers a stable, secure operating system with full compatibility across existing enterprise tools, frameworks and workflows. For organizations adopting hybrid strategies, this ensures a consistent runtime environment, enabling workloads to move seamlessly between Arm-based cloud instances and on-prem infrastructure.

Red Hat OpenShift extends this capability by enabling enterprises to deploy and manage Kubernetes clusters at scale on Arm. AI agents, microservices, and data pipelines can be orchestrated within a unified cloud-native platform, simplifying operations across environments. With Red Hat OpenShift Virtualization support on the AGI CPU, organizations can run virtual machines and containers side-by-side—allowing gradual modernization and migration to Arm without disrupting existing workloads.

Steven Huels, Vice President, AI Engineering, Red Hat, said:

“Red Hat delivers enterprise-grade, open-source platforms that enable innovation across hybrid cloud environments while providing our customers with extensive choice from the application layer to underlying architectures. The expansion of the Arm ecosystem has helped deliver a broader set of customer solutions, and the Arm AGI CPU is poised to continue this trend, powering a next generation of AI-ready infrastructure on Red Hat’s trusted, open foundation.”

Purpose-built silicon for AI infrastructure

Delivering this level of consistency and scalability across environments requires a new foundation at the silicon level.

At the core of this solution is the Arm AGI CPU, Arm’s first system-on-chip (SoC) for datacenter infrastructure, introduced on March 24, 2026. Purpose-built for AI-driven workloads, the AGI CPU is designed to handle everything from scalable inference and orchestration to databases, video processing, and enterprise services.

The AGI CPU features 136 Arm Neoverse V3 cores, 96 lanes of PCIe Gen6, and 12 channels of DDR5 memory running at up to 8800 MT/s — delivering the performance and bandwidth required for next-generation workloads.

Arm-based cloud platforms — including AWS Graviton, Microsoft Azure Cobalt, and Google Axion — have already demonstrated strong performance-per-watt and total cost of ownership advantages. The AGI CPU extends these benefits on premises, enabling organizations to bring cloud-optimized Arm workloads into their own datacenters while maintaining consistency across environments.

Built for the demands of agentic AI

This is critical for agentic AI. Instead of periodic training cycles, enterprises are deploying always-on systems that require continuous inference, real-time orchestration, and efficient data movement at scale.

The AGI CPU is designed to meet these demands. At 300W TDP, it delivers significantly higher efficiency than traditional 500W-class x86 processors. In a standard 36kW air-cooled Open Compute Project (OCP) rack, this enables approximately 8,160 cores per rack, compared to around 4,352 cores for traditional x86 systems—nearly doubling compute density within the same power envelope.

In liquid-cooled environments, such as OCP Open Rack v3, deployments can scale further, supporting up to 336 AGI CPUs and as many as 45,696 cores per rack — delivering up to a fivefold increase in compute density.

While GPU-accelerated platforms lead in AI training, the AGI CPU addresses critical bottlenecks in pre-processing, orchestration, and large-scale inference — key requirements for agentic AI systems.

A validated ecosystem, ready for deployment

The platform is supported by a broad ecosystem of OEM and ODM partners, including Supermicro, Lenovo, and ASRock Rack, providing flexibility in how organizations deploy and scale their infrastructure.

Arm and Red Hat have a long-standing collaboration in the datacenter, and with the AGI CPU, Arm now delivers a fully integrated, enterprise-ready AI stack with Red Hat — combining optimized hardware, trusted software, and efficient performance for production deployment.

Solutions based on this integrated stack are expected to be available in calendar Q4 2026, giving organizations a clear path to deploy next-generation AI infrastructure with confidence.

Learn more about the Arm AGI CPU and the first wave of systems from leading partners here.