Scale unveils chemistry test developed with NIST's AI standards center

Scale AI unveiled DualChem on Wednesday, a chemistry benchmark developed with NIST's Center for Advancing Innovation and Standards for Super Intelligence to help government evaluators assess AI assistance and safeguards, FedScoop reported. The test contains 600 expert-curated questions covering molecular design, synthesis procedures and information management. It focuses on realistic questions whose answers could have beneficial or harmful applications, rather than explicit requests for harmful information. Scale said testing roughly 30 frontier models found generally 50% accuracy, with top performers at roughly 67%. Models often answered even when they recognized a question's potential for harmful use, and their severity assessments varied significantly from expert judgments.