Preprint reports faster, better-aligned AI evaluation of radiology reports

Researchers report that lightweight fine-tuning of Qwen3-30B improved agreement with radiologists by up to 25% and reduced latency by 37.2x in a preprint evaluating AI systems that judge radiology reports. The study used two expert-annotated datasets spanning multiple imaging modalities and anatomical regions. Their proposed evaluation metric, VERT, improved Kendall's correlation with radiologist judgment by up to 26% over GREEN, the strongest existing metric tested. Although the automated metrics correlated with expert ratings overall, they tended to miscount errors such as omissions.