CPU Bottleneck — The Hidden Constraint on AI Scaling

Vector 1: RL environment simulation

Reinforcement learning training loops work by generating candidate outputs, then scoring them against an environment. The environment — opening files, editing CAD models, submitting to websites, running code — executes on CPUs, not GPUs or ASICs. As RL environments grow more complex to train more capable models, CPU demand scales with that complexity.

This creates a bottleneck distinct from the GPU shortage. A training cluster with thousands of GPUs for forward and backward passes also needs a proportional fleet of CPUs running the scoring environments. Scaling GPU capacity without matching CPU capacity for RL environments creates a lopsided cluster that cannot fully utilize its accelerators.

Vector 2: inference diffusion into applications

Every token of code, structured data, or actionable output that an AI model generates eventually runs somewhere. That “somewhere” is overwhelmingly CPU-based cloud infrastructure: web servers, application runtimes, database queries, CI/CD pipelines, data processing jobs.

As AI-generated code proliferates — and as agents move from generating suggestions to executing multi-step workflows — the downstream CPU footprint grows in proportion. A model that writes and deploys a microservice consumes GPU tokens during inference but creates ongoing CPU demand for the service itself.

Why this matters for investment

The GPU shortage is well-understood and priced into public markets (Nvidia, AMD, Broadcom). The CPU shortage from these two vectors receives far less attention:

Both vectors compound. Better models need harder RL environments (more CPU). Better models also generate more code that runs on more CPUs. The result is that CPU demand grows on both sides of the training-inference divide.

Dylan Patel / SemiAnalysis flagged this as one of the least-discussed supply chain constraints in April 2026, noting that CPU shortages were already appearing but receiving little coverage relative to GPU supply discussions.

Related Notes

• EUV Lithography as the Binding Constraint on AI Scaling — the structural ceiling on chip manufacturing that affects CPUs and GPUs alike
• AI Memory Crowding - HBM Eats Consumer Device Budgets — another under-discussed supply chain constraint where AI demand crowds out other uses
• RL Scaling Follows Pre-Training - The Generalization Inflection Ahead — the RL scaling trend that drives Vector 1 demand
• CUDA Programmability Moat - Why Flexibility Beats Optimization — why GPUs dominate the accelerator side while CPUs handle the environment side