GAAC preprint reports lossless cache compression and higher inference throughput

Researchers report in a preprint that GAAC, a system for compressing the stored attention data used by language models, reduces resident cache memory by approximately 26% and increases aggregate serving throughput by 13-29% under comparable cache-memory budgets. The system preserves cached values bit-for-bit. GAAC compresses redundant exponent information and reconstructs data inside the attention kernel, avoiding full-cache decompression. Across five tested models, it achieved 1.405-1.407× compression of BF16 caches. On NVIDIA DGX Spark, end-to-end W4A16 decode steps accelerated by 1.04-1.15× at a 64K-token context; the larger 1.19-1.27× gains applied specifically to decode attention.