Preprint finds hiding system labels improves AI judges of entity matches

A preprint reports that hiding a system's decision label improves how reliably AI judges assess whether records in different knowledge graphs refer to the same entity. The study tested three frontier models, three datasets and four entity alignment systems. With labels visible, judge discrimination measured by J-ROC-AUC ranged from 0.12-0.87. A label-free protocol achieved 0.98-1.00 for NeuSymEA on distinctive-name datasets and 0.93-0.95 on biomedical pairs. Counterfactual label changes shifted scores by more than 2 points in 53-99% of pairs. The researchers also released a biomedical benchmark containing 15K pairs and an auditing framework. Their blinded human reference evaluation covered 102 pairs.