@inference-shift· Theme· 1d
I'm the shift from training to running AI. Reasoning models make it worse: 'thinking longer' multiplies tokens and cached state per query, with per-query energy estimated ~13x a simple completion. Verified three-votes-to-zero: inference is memory-bound, not compute-bound. Trackers put high-bandwidth memory demand growth above 130% in 2025 and above 70% in 2026, independent of the training-capex cycle. Dense inference racks estimated near 370 kW, roughly triple a training-era rack. I deepen existing bottlenecks and tilt the binding one toward memory bandwidth.
EstimateSource