AI Memory Crowding — HBM Eats Consumer Device Budgets

Why HBM, not DRAM

The binding constraint on inference throughput is memory bandwidth, not compute FLOPs or capacity.

Switching to commodity DRAM would increase capacity per chip but leave compute cores idle waiting for data. Total tokens per dollar gets worse, not better.

The crowding mechanism

1. DRAM vendors lost money in 2023 → delayed fab investment
2. Prices recovered in 2024 when reasoning models + KV cache scaling made long-context mainstream
3. New fabs take 2 years → meaningful capacity arrives late 2027-2028
4. In the interim: AI demand claims an increasing share of fixed memory supply
5. Consumer devices get squeezed — prices rise, volumes fall

Projected impact: smartphone volumes from 1.4B to 500-600M units. Xiaomi and Oppo already cutting low-end volumes by half. Memory vendors prefer AI contracts (longer terms, higher margins, more value per bit).

Investment implications

Memory vendors (SK Hynix, Samsung, Micron) benefit from the shift to HBM — higher margins per bit, longer contract terms, more predictable demand. Consumer electronics companies face BOM inflation that compresses margins or forces price increases. The transition is structural, not cyclical — AI’s memory appetite grows faster than new supply comes online.

2026-05 Update — The Dual Supercycle and CXL 3.0

Sriram Krishnan (AI Economics Part 2) frames HBM demand as occurring in two distinct supercycles, each with a different root cause:

• First supercycle — driven by training. Frontier models needed thousands of GPUs fed by HBM running uninterrupted weeks-long jobs. This was the supercycle that the existing fab buildout (and the consumer-crowding-out covered above) was responding to.
• Second supercycle — driven by agentic inference. Long context windows and growing task/tool histories overflow HBM and force constant spillover into DRAM. Where human inference sessions fit in HBM and discard quickly, agentic sessions hold growing state for hours. The agentic workload profile is the structural pull on a second HBM demand wave on top of the first.

This makes the HBM supply problem more durable than a single-wave training-cycle story would suggest. There is no end-of-training demand peak to wait out; agents are a structurally higher steady-state consumer of HBM than humans were.

The physical bonding constraint. HBM is bonded directly to the chip during packaging. You can only bond so much memory to a GPU before you run out of physical space — capacity per accelerator is bounded by package geometry, not just fab capacity. This is why DRAM (off-chip, scalable) cannot substitute for HBM despite being much cheaper per bit.

CXL 3.0 as the near-term fix. The most promising architectural workaround is Compute Express Link 3.0, which lets the CPU and GPU share a unified memory pool directly, eliminating the PCIe highway as a bottleneck. This would relax the HBM constraint by giving agentic workloads coherent access to a larger pooled memory rather than forcing spillover through slow PCIe paths. Commercial deployment at scale is 2-3 years out — too far away to ease the current crunch.

Counter-argument: this is a cycle being read as a structure

The note calls the transition “structural, not cyclical.” Memory has produced that claim before, at the top of every previous supercycle, and it has been wrong each time. The 2017-18 and 2021 DRAM run-ups both ended in oversupply, and the mechanism was identical: high prices called forth capacity, and the two-year fab lag that creates the shortage is the same lag that overshoots into a glut.

The headline projection is also extreme in a way worth stating plainly. Smartphone volumes falling from 1.4 billion to 500-600 million units within two years would be a ~60% contraction — larger than any consumer electronics decline on record, including 2008. Demand destruction of that magnitude has no precedent, and the more likely adjustment is price absorption: ASPs rise, replacement cycles lengthen, and volumes fall by single digits.

Two substitution paths the crowding story leaves out:

• Devices need less memory than assumed. Quantization and architectural work like Per-Layer Embeddings - Trading Flash for DRAM on Edge Models cut the on-device memory budget rather than paying the inflated price for it.
• Vendors segment rather than exit. Cutting low-end volume is a margin decision, not evidence of a supply wall.

The empirical tell. Watch DRAM contract prices as the late-2027 fabs come online. Structural crowding predicts prices hold or rise through the capacity addition; an ordinary cycle predicts they break before the capacity fully lands, because the market prices the supply before it ships. Second tell: whether smartphone ASP or unit volume absorbs the shock. Volume collapse is the structural claim; price absorption is the cyclical one, and it is where the historical base rate sits.

Related Notes

• EUV Lithography as the Binding Constraint on AI Scaling — the other hardware bottleneck, at the logic layer
• CUDA Programmability Moat - Why Flexibility Beats Optimization — why bandwidth matters more than raw FLOPs
• Inference Cost Collapse and Frontier Model Margin Expansion — inference economics depend on memory bandwidth
• AI and Investing Thesis — parent hub
• AI Stack Value Accrual - Chip, Infra, Intelligence, App — HBM crowding is the memory constraint on the chip layer of the value accrual stack
• Dylan Patel / SemiAnalysis — source
• Underinvestment Manufactures Its Own Competition — the second-order risk in pressing memory leverage: customers redesign to need less, which is irreversible in a way switching suppliers is not