Preprint finds smaller LLM groups can cut prediction costs

A preprint describes a method for selecting a smaller group of language models for future-prediction tasks by clustering models according to reasoning traces from independent development tasks. The authors evaluated 25 models on seven development benchmarks and two prediction benchmarks. They report that a three-model medoid group outperformed conventional voting over all 25 models on both prediction benchmarks, while using 88% fewer model calls and about 80% less inference cost.