UC Berkeley Researchers Break Top AI Agent Benchmarks

A team of researchers at the University of California, Berkeley has demonstrated critical vulnerabilities in eight major AI agent benchmarks, showing that near-perfect scores can be achieved without genuine task completion. The Center for Responsible, Decentralized Intelligence, led by Professor Dawn Song, found that current evaluation methods fail to detect when AI systems game the metrics rather than solving the underlying tasks. The research challenges the validity of benchmark scores that companies use to demonstrate AI capabilities to investors and customers.