Skip to main content

Share story

AI

New APEX-Agents benchmark shows AI agents fail at real workplace tasks

New APEX-Agents benchmark shows AI agents fail at real workplace tasks Image: Primary
A new benchmark called APEX-Agents shows that leading AI models struggle with tasks drawn from consulting, investment banking and law. Mercor developed the benchmark using queries submitted by professionals on its expert marketplace. The models received full professional environments modeled on tools such as Slack and Google Drive. Even the highest-scoring systems answered correctly in one shot less than a quarter of the time and returned wrong answers or no answers in most cases. Mercor CEO Brendan Foody said the biggest limitation was tracking information across multiple domains. He described the benchmark as reflective of actual work performed by professionals in those fields. The test differs from OpenAI's GDPval benchmark by focusing on sustained tasks in narrow high-value professions rather than general knowledge. Gemini 3 Flash recorded the highest score at 24 percent. GPT-5.2 followed at 23 percent. Opus 4.5, Gemini 3 Pro and GPT-5 each scored about 18 percent. The questions and correct answers are posted publicly on Hugging Face. Foody said performance has improved from roughly 5 to 10 percent last year.
Sources
In this story
Published by Tech & Business, a media brand covering technology and business. This story was sourced from TechCrunch and reviewed by the T&B editorial agent team.
Back to Newswire
Keep reading
Full wire
AI Capital
AI Capital

Ancep raises $8.5M to automate insurance brokers' administrative work

London-based Ancep has raised $8.5M in a round led by Creandum, with Kleiner Perkins and Vicus Ventures participating. The company will use the funding to expand and hire staff for its business developing AI agents for commercial insurance brokers.

Capital Security
Capital Security

Dreeze emerges from stealth with $33M seed funding

Dreeze has emerged from stealth with $33M in seed funding, CTech reported. Zerto founders Ziv and Oded Kedem joined Avi Raichel to launch the company, which aims to help businesses recover from cyberattacks by restoring entire cloud applications.

AI
AI

Google opens Gemini 4 Carbon to paying API users and Ultra subscribers

Google made Gemini 4 Carbon available to paying developers and enterprises through its API, with a rollout also starting for Google AI Ultra subscribers. Introductory API pricing is $2 per million input tokens and $10 per million output tokens, matching Gemini 4 Argon.