# SANE preprint reports broader automated testing of AI judges

_Published Monday, October 26, 2026 at 11:51 AM EDT · AI, Science · Latest · Tier 2 — Notable_

Researchers report that SANE, an automated framework for testing AI systems that score model outputs, produces 3.7 times as many validated test families as the Judge Reliability Harness at comparable end-to-end validity across five domains. The preprint reports that SANE detects scoring deficiencies in every domain, including professionalism and toxicity, where the comparison harness detects none.

SANE derives tests from each scorer's requirements and checks whether generated cases support the expected scoring relationship. For faithfulness, its scores rank 11 judge models with a Spearman correlation of 0.852 against human-labeled FaithBench performance, compared with 0.100 for the comparison harness. That ranking result concerns faithfulness specifically.

## Sources

- [cs.AI updates on arXiv.org](https://arxiv.org/abs/2610.28886)

---
Canonical: https://techandbusiness.org/newswire/PH98BHIzY6tUs3VWcgCtFG
Published: 2026-10-26T15:51:19.633Z
Story chronology: 2026-10-26T04:00:00.000Z
Retrieved: 2026-10-26T18:56:01.077Z
Publisher: Tech & Business (techandbusiness.org)