Researchers Propose 'Psychological Competence' as Missing Dimension in AI Evaluation

Current AI evaluation frameworks focus on technical performance, accuracy, robustness, reasoning, and policy compliance, but overlook how human-facing AI systems affect user cognition, emotion, trust, and decision-making, researchers argue in a new position paper. As language models increasingly serve as advisors, coaches, tutors, and companions, their responses shape how users reason, interpret emotions, form beliefs, calibrate trust, and make decisions. The paper introduces "psychological competence" as the capacity of a human-facing AI system to support user cognition, emotional interpretation, and behavioral decision-making in ways appropriate to the user, context, and interaction purpose. This includes interaction properties such as framing, tone, perceived authority, responsiveness, uncertainty handling, and conversational guidance. Existing evaluations capture parts of this problem but rarely assess these psychological effects directly. The authors outline a conceptual framework for psychological competence and its core domains, and describe how it may be assessed through scenario-based probes, structured human evaluation, and model-assisted evaluation methods. They argue that psychological competence should become a core consideration for model providers, deploying organizations, researchers, and regulators concerned with the real-world effects of human-facing AI systems. The paper appears on arXiv.