SPLIT preprint reports improved allocation of AI inference budgets

Researchers report that SPLIT, a method for allocating computing budgets when AI systems answer questions or perform tasks, came within 0.02-0.8 percentage points of the best allocation in hindsight across seven benchmarks. The preprint covers 26 models, 175 agent systems and six kinds of answer-checking systems. SPLIT uses a small labelled pilot to choose how many candidate answers to generate, which models to use and how much to spend checking answers. Its formula accounts for checkers accepting wrong answers and making correlated mistakes. Every fixed allocation rule tested lost at least 2.5 points in some setting; the best allocation used for comparison was determined in hindsight.