AI Monitoring (Production)
Ongoing surveillance of AI systems in production to detect drift, failures, and compliance issues.
Definition
AI Monitoring is the continuous observation of AI systems in production environments. Unlike traditional software, AI systems can degrade or change behavior over time without code changes. Key monitoring areas:
- Performance monitoring: Tracking accuracy, latency, and error rates
- Data drift detection: Identifying when input data distributions change
- Model drift detection: Detecting degradation in model predictions
- Fairness monitoring: Ongoing assessment for emerging bias
- Incident detection: Identifying AI failures or unexpected behaviors
- Usage monitoring: Tracking how AI is being used and by whom
Regulatory requirements for monitoring include EU AI Act post-market monitoring obligations and financial services expectations under SR 11-7.
Sources
- •EU AI Act Article 72
- •NIST AI RMF
Related Terms
Post‑market monitoring
Ongoing oversight of AI system performance and incidents....
AI Testing and Validation
Systematic evaluation of AI systems for performance, safety, fairness, and compliance....
post-deployment testing
Monitoring and assessments after release....
AI system operation and management
Deployment, monitoring and maintenance of operational AI models....
Post-Market Monitoring System
Provider activities collecting experience from deployed AI systems....