HypercubeQuant: early notes from an A100 KV-cache experiment
HypercubeQuant is an in-progress inference-time memory experiment for transformer serving on a single NVIDIA A100. This is an early teaser note: what it is, where it sits in the space, what the current evidence actually…
+1 topic