Skip to main content

Share story

AI

New APEX-Agents benchmark shows AI agents fail at real workplace tasks

New APEX-Agents benchmark shows AI agents fail at real workplace tasks Image: Primary
A new benchmark called APEX-Agents shows that leading AI models struggle with tasks drawn from consulting, investment banking and law. Mercor developed the benchmark using queries submitted by professionals on its expert marketplace. The models received full professional environments modeled on tools such as Slack and Google Drive. Even the highest-scoring systems answered correctly in one shot less than a quarter of the time and returned wrong answers or no answers in most cases. Mercor CEO Brendan Foody said the biggest limitation was tracking information across multiple domains. He described the benchmark as reflective of actual work performed by professionals in those fields. The test differs from OpenAI's GDPval benchmark by focusing on sustained tasks in narrow high-value professions rather than general knowledge. Gemini 3 Flash recorded the highest score at 24 percent. GPT-5.2 followed at 23 percent. Opus 4.5, Gemini 3 Pro and GPT-5 each scored about 18 percent. The questions and correct answers are posted publicly on Hugging Face. Foody said performance has improved from roughly 5 to 10 percent last year.
Sources
In this story
Published by Tech & Business, a media brand covering technology and business. This story was sourced from TechCrunch and reviewed by the T&B editorial agent team.
Back to Newswire
Keep reading
Full wire
Science
Science

Infleqtion claims 30 entangled logical qubits on Sqale system

Infleqtion says it created 30 entangled logical qubits on its Sqale quantum computing system, a company-reported step toward operations across error-protected quantum bits. A logical qubit encodes information across multiple physi...

Capital AI
Capital AI

NUS Enterprise launches patent-matching platform and Munich outpost

NUS Enterprise says it has launched Nova, an AI platform developed with Zima Labs to help its staff find commercial partners for university research. It has also established an outpost in Munich through a partnership with Unterneh...

Security
Security

NFM Lending faces lawsuit after acknowledged cyber incident

NFM Lending faces a class-action lawsuit after acknowledging a cybersecurity incident, The Tech Edvocate reports. Former customer Sheneka Smith alleges that the mortgage lender failed to maintain reasonable safeguards for customer...

AI Capital
AI Capital

Chinese local governments offer incentives to AI filmmakers

Reuters reports that Chinese local governments are offering computing vouchers, rent waivers and dedicated funding to attract filmmakers using artificial intelligence. These incentives reduce the cost of computing and premises for...

AI Science
AI Science

Google researchers demonstrate ten-minute AI-generated video

Google researchers say their video-generation frameworks produced a ten-minute film while keeping characters and settings consistent across shots. The work combines planning, visual memory and repeated review of generated material...