AutoCompress preprint reports 74B model serving at 128K context on one B200

Researchers report in an AutoCompress preprint that they compressed Nemotron 3.5 Super from 120B to 74B parameters for BF16 serving at 128K context on a single B200 GPU, retaining 95% of its mean benchmark score. The framework searches model architectures using latency, throughput and memory constraints rather than parameter count alone. It adjusts both overall and individual-layer structure, guided by serving-engine cost models checked against end-to-end measurements. Across eight benchmarks, the authors report average accuracy gains over Minitron of 6.57 percentage points on RTX 5070, 2.35 on RTX 5080 and 0.57 on H100. Those comparisons cover specific hardware and workloads.