Back to Newswire
AI Products

RuBench Tests Coding Agents on Russian-Language Tasks From Real Repositories

arXiv logo Image: Primary
A new benchmark called RuBench 1.0 evaluates product-grade coding agents on 25 repository-level tasks mined from recent fix commits in five live open-source repositories (aiohttp, aiogram, Laravel, NestJS, Fastify) spanning Python, PHP, TypeScript, and JavaScript. Each task is specified natively in Russian, written from scratch in the style of an actual customer request rather than translated from English. Tasks are judged
Sources
cs.AI updates on arXiv.org
Published by Tech & Business, a media brand covering technology and business. This story was sourced from cs.AI updates on arXiv.org and reviewed by the T&B editorial agent team.
Related Stories
AI Infrastructure

Hidden-State Probes Predict LLM Agent Failure Early, Saving Up to 47% Inference Compute

cs.AI updates on arXiv.org
AI Infrastructure

o3-mini Achieves Higher Accuracy Than o1-mini Without Longer Reasoning Chains

cs.AI updates on arXiv.org
AI

FirstResearch Framework Makes AI-Generated Scientific Questions Auditable

cs.AI updates on arXiv.org
AI Security

HARC Couples Harmfulness and Refusal Directions for Stronger LLM Safety Alignment

cs.AI updates on arXiv.org