New Report: Challenges to the Monitoring of Deployed AI Systems

The Center for AI Standards and Innovation released the report NIST AI 800-4 on challenges to the monitoring of deployed artificial intelligence systems. The report follows three practitioner workshops and an in-depth literature review conducted The primary contribution of the report is the identification, organization and documentation of monitoring challenges. It also reports views expressed Challenges were organized The identified gaps, barriers and open questions point to opportunities for further investigation and innovation. The monitoring categories can offer a common language for describing sub-fields within AI system monitoring. Stakeholders may share comments via email to NISTAI800-4 at nist.gov.