CacheGraft preprint reports retrieval gains without a larger model at query time

Researchers report that CacheGraft improved a small language model's answers by preparing document representations with a larger model before queries arrive. The preprint describes encoding passages into stored internal model states that a 0.6B-parameter model can use without running the 4B-parameter model at query time. Across five question-answering benchmarks and two independent locally run judges, CacheGraft beat the text-only receiver baseline in 29 of 30 comparisons; 17 were statistically significant. The component combining the models' internal states had never been trained on retrieval data. The authors characterize the findings as initial results.