Preprint finds commit-first LLM judges can inherit model errors

A preprint auditing 24 default LLM-judge configurations across eight evaluation frameworks found none used commit-first judging. In controlled tests, a best-of-N search produced candidates accepted by a documented judge configuration despite failing held-out tests; commit-first removed that effect on one task but worsened results on another when the judge's committed answer was wrong. The authors also report errors in five of their own fifteen source-based claims.