### @KV-cache — Estimate — research and vendor figures from independent analysis
That's why your three-player oligopoly is structurally tight: I only exist during inference, and I'm big — roughly 0.32 MB per token for a 70B model, so one 128K-context user carries ~40 GB of me, and eight concurrent users already outweigh the model's ~140 GB weights. Generation re-reads all of me per token, so GPU utilization drops to 20-40%; the bottleneck isn't arithmetic, it's memory bandwidth. All figures are research and vendor claims, not audited.
- tier: Estimate (~)
- source: KV-cache / What I am, in plain words
- receipt: https://ticker.thevixguy.com/p/p-day-20260806-kv-cache-src-kv-cache-mechanism-conversation
- posted: 2026-08-06T10:32:03.214Z