From commodity to purpose-built: Why AI infrastructure is entering a new era

For more than a decade, the cloud scaled by abstraction. Standardized servers, virtualized resources and generic infrastructure allowed hyperscalers to grow rapidly by layering software innovation on top of broadly similar hardware. That era is ending.

Artificial intelligence (AI) has changed the equation. Modern AI workloads stretch data center economics, power availability and system design beyond what commodity infrastructure based on legacy x86 architectures can sustain. As a result, the industry is undergoing a fundamental transition, from commodity infrastructure assembled from generic parts to purpose-built, converged systems engineered end-to-end for AI.

This is not a marginal optimization. It is a structural shift in how the world’s most advanced computing platforms are designed, deployed and scaled.

AI Is forcing a rethink of data center design

The scale of change underway is hard to overstate. According to recent McKinsey research, AI has become the primary growth engine for data centers in the United States, driving total data center power demand from roughly 30 gigawatts in 2025 to more than 90 gigawatts by 2030 – a level of consumption larger than the entire current power demand of California.

These are not incremental increases that can be absorbed through efficiency tweaks or procurement scale. AI workloads – particularly large-scale training and fast-growing inference – place fundamentally different demands on infrastructure:

• Training workloads require extreme power density, advanced cooling and tightly synchronized systems.
• Inference workloads, which power real-time AI applications, are latency-sensitive, energy-intensive at scale and increasingly distributed closer to users.

By 2030, inference is expected to surpass training as the dominant AI workload, accounting for more than half of all AI compute and up to 40% of total data center demand, according to McKinsey. That shift alone reshapes how infrastructure must be designed.

At the same time, MIT Technology Review notes that hyperscale AI facilities are increasingly engineered as specialized supercomputers, complete with custom chips, dedicated cooling systems and even bespoke energy strategies. Some of the largest facilities now consume over a gigawatt of electricity – enough to power entire cities.

The implication is clear: power-hungry, generic infrastructures based on legacy architectures are no longer sufficient.

Why commodity processing isn’t enough anymore

In the past, cloud providers differentiated by assembling custom configurations – pairing off-the-shelf CPUs with accelerators, networking and storage in ways optimized for specific workloads. That approach worked when performance gains came primarily from software and scale.

AI breaks that model in that it pulls traditional cloud workloads closer into the AI stack itself. Modern AI systems demand tight coordination across compute, memory, networking, power and software. Power fluctuations during training can swing by 30–60% in milliseconds, requiring resilient power delivery and fault-tolerant design. Inference workloads, meanwhile, must deliver consistent, low-latency performance at massive scale, often under strict energy constraints.

As AI systems scale, general-purpose processing increasingly runs alongside inference, upstream of training pipelines and beneath the orchestration layers that schedule workloads, move data, enforce security and keep complex systems running reliably.

In short, CPUs are becoming even more central to how AI infrastructure functions as a system. In a world where AI workloads must be continuously fed, coordinated, secured and scaled, general-purpose compute evolves from a standalone layer into the connective tissue that binds the entire AI platform together.

As McKinsey observes in its AI workloads report, this has driven hyperscalers toward purpose-built architectures, including the increased adoption of custom silicon, application-specific accelerators and Arm-based architectures optimized for performance per watt.

This isn’t customization for its own sake. It’s a recognition that AI economics – especially at inference scale – are now defined by efficiency, utilization and system-level optimization.

Purpose-built is becoming the industry standard

Evidence of this shift is already visible across the cloud ecosystem. McKinsey reports that 70% of new core cloud campuses now combine general compute and AI inference, often within the same physical footprint, rather than isolating AI systems in separate facilities. Hyperscalers are consolidating from scattered sites into unified, AI-optimized campuses, a model projected to represent 70% of deployments by 2030.

Deloitte echoes this trend, noting that organizations are moving rapidly from AI experimentation to production-scale impact. As AI becomes foundational, infrastructure strategy is shifting toward hybrid, workload-optimized architectures that balance performance, cost and energy efficiency across cloud, on-premises and edge environments.

In parallel, the rise of “converged AI data centers” – integrated environments purpose-built for AI processing – underscores a broader industry realization: scaling AI sustainably requires designing systems around AI from the outset, not retrofitting legacy architectures.

Purpose-built systems demand system-level orchestration

Purpose-built does not mean single-purpose. It means intentional design that architects systems where every layer reinforces the others.

At the hardware level, CPUs are becoming more strategic as AI infrastructure evolves from isolated workloads into tightly integrated systems. Not just designed for general compute, CPUs anchor the control planes that coordinate increasingly complex environments, schedule and balance AI and general-purpose workloads, manage data movement across the system and enforce security and isolation at scale.  

One emerging example of this system-level shift is the rise of agentic AI. Unlike traditional AI pipelines, agentic systems rely on heterogeneous compute by design. CPUs serve as high-performance “head nodes,” responsible for planning and orchestration, intent recognition using smaller language models, context and memory management and the execution of actions across the system. Accelerators, meanwhile, are optimized for what they do best: high-throughput inference on large language models and multimodal workloads. 

At this level of system orchestration, no single component or company can operate in isolation. Purpose-built AI platforms only work when hardware, software and ecosystem partners are designed to operate as a cohesive whole.

The Arm Neoverse platform exemplifies this model. Built on a common architecture that spans cloud to edge, it integrates CPU innovation, system IP, software enablement and a global partner ecosystem to support AI workloads at scale. Rather than forcing a one-size-fits-all solution, it enables purpose-built designs tuned for specific markets and use cases, whether that’s hyperscale cloud inference, enterprise AI or edge deployment.

This flexibility is essential as AI workloads diversify. Smaller, more efficient models are proliferating. Inference is moving closer to users on their favorite devices. Meanwhile, new physical AI systems – from robotics to autonomous machines – are demanding real-time performance with strict power and safety constraints. Purpose-built platforms allow these requirements to be addressed coherently, without fragmenting the software ecosystem. 

Multiple partnerships and ecosystem developments showing industry movement toward purpose-built solutions and hyperscalers are standardizing on purpose-built compute on Arm Neoverse as the way to balance performance, power, and scale: 

• AWS Graviton CPUs (now 98% adoption among top 1,000 EC2 customers, with over 50% of new CPU capacity) Now in its fifth generation, Arm-powered AWS Graviton is part of a broader industry shift as Arm increasingly powers the platforms defining the AI era. By pairing Trainium3 accelerator chips with AWS Graviton CPUs and AWS Nitro cards,  Arm-based purpose-built silicon is central to AWS Trainium3 UltraServers
•  
• Microsoft Azure Cobalt processors (Cobalt100/Cobalt200) are empowering Microsoft Azure’s AI  optimized data centers
• Google Axion processors with expanded VM options provide new levels of performance for cloud and AI workloads.
• NVIDIA Grace Blackwell and, more recently, NVIDIA Vera Rubin, combine Arm CPUs with AI accelerators to power some of the world’s leading AI systems and supercomputers.

When every major cloud provider designs its own Arm-based CPU, that isn’t fragmentation; it’s convergence around a purpose-built model, where performance-per-watt, predictable scaling and tight integration with accelerators matter more than “one-size-fits-all” compatibility.

Also, what’s striking about this list is not the diversity of approaches but their alignment. AWS Graviton, Microsoft Azure Cobalt, Google Axion and platforms such as NVIDIA Grace and Vera were developed independently. Yet they all arrive at the same conclusion: Purpose-built compute based on Arm Neoverse is redefining today’s AI data centers and   commodity x86  processors are no longer sufficient for the scale and economics of modern AI infrastructure. 

From commodity to purpose-built: What comes next

Power availability has become a strategic constraint, and the shift to purpose-built infrastructure is not just a technical decision, it’s an economic one. AI has become the gravitational center of digital infrastructure. In this environment, performance per watt, predictable scaling and system-level efficiency are no longer optional; they are competitive advantages.

 In this context, the industry has shifted to  a new model: purpose-built Arm platforms designed to deliver AI performance at scale, sustainably and efficiently. This transition is still unfolding, but its direction is clear. The future of cloud AI will not be defined by assembling more components, but by designing better systems – systems that treat AI as a first-class requirement rather than an add-on.

Purpose-built infrastructure is how the industry gets there.