QRAE preprint reports lower token use and latency for search agents

Researchers report that QRAE, a method for selecting which retrieved evidence search agents retain, reduced average end-to-end latency by 30% and reported token usage by 45% across multiple benchmarks and reasoning models. The preprint also reports an average task-success improvement of 2.35 percentage points. QRAE uses model attention to select intact evidence chunks for subsequent reasoning. It caches uncertain chunks for possible reevaluation rather than including them immediately. The method requires neither an additional text-generation pass to compress evidence nor changes to the attention operator. The authors report that the benefits also extend to a model with nearly 100B parameters.