Study finds cheaper claim verification carries higher unsupported-claim acceptance

Researchers report in a preprint that combining a cheaper claim verifier with a frontier language model scored 74.10 at USD 2.92 per thousand claims, against 73.83 at USD 8.93 for the frontier model alone. The system sends some claims to the frontier model and lets the cheaper verifier decide the rest. Testing used 495 human-labelled claims. The cheaper verifier alone accepted 23.2% of unsupported claims, compared with the frontier model's 14.1%, despite statistically indistinguishable overall scores and a 187-fold cost difference. Randomly choosing which claims to escalate performed as well at the same budget in a control using in-sample thresholds. Against the cheaper verifier alone, the combined system cost 61-fold more for a 0.83-point gain, with a 95% confidence interval of [-3.36, 4.86], and accepted more unsupported claims.