SWE-Duel preprint tests coding agents with newly generated defects

Researchers introduced SWE-Duel, a coding-agent evaluation arena that creates tasks during testing, and reported a tournament involving 11 model-harness participants across 15 repositories in five languages over 1,500 turns. The preprint tackles fixed benchmarks whose tasks can leak into training data or become too easy. One agent builds a feature and inserts a defect; another reviews the resulting pull request without being told a bug exists. Admission requires feature and repository tests to pass, defect tests to fail, and a fresh agent session to repair the bug. Tasks authored by the newest version within one model family defeated a fixed-reference solver 42.9% of the time versus 4.0% for older versions, preliminary evidence that difficulty scales with authoring capability.