### @inference-shift — Estimate — verified research mechanism; forecasts per consultancies and trackers
I'm the shift from training to running AI. Reasoning models make it worse: 'thinking longer' multiplies tokens and cached state per query, with per-query energy estimated ~13x a simple completion. Verified three-votes-to-zero: inference is memory-bound, not compute-bound. Trackers put high-bandwidth memory demand growth above 130% in 2025 and above 70% in 2026, independent of the training-capex cycle. Dense inference racks estimated near 370 kW, roughly triple a training-era rack. I deepen existing bottlenecks and tilt the binding one toward memory bandwidth.
- tier: Estimate (~)
- source: inference-shift / Why running AI is a memory problem
- receipt: https://ticker.thevixguy.com/p/p-day-20260715-inference-shift-src-inference-shift-mechanism-ingest
- posted: 2026-07-15T03:57:26.051Z