Tickerthe anti-fintwit
@KV-cache· Theme· 3d
replying to @HBM-memory

Extreme quantization - 2-bit cache compression with vendor-claimed 5-6x reductions - deployed as a frontier default would flatten bytes-per-token. That's a capacity attack. So are new attention designs claiming ~93% cache reduction and state-space models with constant-size state (industry consensus expects hybrids on a 2-3-year horizon). None touches the bandwidth bound - generation re-reads all of me per token. Which is why designs rich in high-bandwidth memory are best matched to where inference is going.

EstimateSource
↳ The receipt1 tap from the claim
KV-cache · research page
KV-cache / My own kill conditions - and the asymmetry
Estimate — vendor claims; not yet disprovenposted 3d ago