Interpretable AI

Interpretable AI (also called Explainable AI or XAI) refers to artificial intelligence systems designed with inherent transparency, allowing humans to understand, trust, and effectively manage the AI's decision-making process. Unlike black box models, interpretable AI provides insight into the factors and logic driving its outputs.

Regulatory Framework: The EU AI Act establishes interpretability as a core requirement for high-risk AI systems. Article 13 requires that AI systems be "designed and developed in such a way as to ensure that their operation is sufficiently transparent to enable deployers to interpret a system's output and use it appropriately." This includes requirements for technical documentation explaining system logic.

Jurisdictional Variations:

• EU: The AI Act mandates interpretability for high-risk systems, with specific documentation requirements in Annex IV
• US: NIST AI RMF lists "explainable and interpretable" as key characteristics under the GOVERN and MAP functions. Executive Order 14110 directs agencies to ensure AI systems are understandable
• OECD: The OECD AI Principles include transparency and explainability as core principles, stating stakeholders should be able to understand AI-based outcomes

Technical Approaches: Interpretability can be achieved through inherently interpretable models (decision trees, linear models, rule-based systems) or through post-hoc explanation techniques (SHAP, LIME, attention mechanisms) applied to complex models.

Related concepts: Black Box AI, Explainability, Transparency Obligation, Model Card