Rubricon preprint reports gains from automatically generated scoring criteria

A preprint introduces Rubricon, a system that generates and refines criteria for scoring AI answers without human preference labels, annotated rubrics or training the model that generates them. Across five benchmarks, its scores showed 9% greater alignment with expert-authored rubrics than the best competing synthetic method. The system samples candidate rubrics, selects one, adds missing criteria and removes shallow or misaligned criteria, using observed pass rates to guide refinement. On HealthBench, training Qwen3.5-9B with its rubrics produced 4.5% better performance than training with rubrics from RRD, the strongest alternative tested.