ISO/IEC AI Functional Safety Systems

Overview

ISO/IEC TR 5469:2024 is a pivotal Technical Report that addresses the critical intersection of artificial intelligence (AI) and functional safety, providing essential guidance for the responsible and safe integration of AI systems into safety-critical applications. Published in January 2024, this document fills a significant gap in the existing standardization landscape by offering a comprehensive framework for understanding and managing the unique challenges posed by AI technologies in environments where system failures could lead to severe consequences for people, property, or the environment. It serves as a crucial bridge between established functional safety standards, such as the widely adopted IEC 61508 series, and the rapidly evolving field of AI. The report aims to foster awareness among developers, integrators, and users of AI systems regarding the specific properties, associated risk factors, available methods, and potential constraints of AI technologies when deployed in safety-related functions.

The significance of ISO/IEC TR 5469:2024 lies in its direct contribution to building trust in AI systems, particularly in high-stakes domains such as healthcare, autonomous vehicles, industrial automation, and aviation. By providing a structured approach to assessing and mitigating risks, the Technical Report enables organizations to develop and deploy AI applications that not only deliver innovative benefits but also adhere to stringent safety requirements. It acknowledges that while AI offers immense potential for efficiency and advanced capabilities, its non-deterministic and data-driven nature necessitates specialized considerations for functional safety. This document is a foundational step towards ensuring that AI technologies can be responsibly adopted across a broader spectrum of application domains, thereby unlocking their full societal benefits while rigorously addressing ethical and trustworthy concerns.

Definitions

For the purposes of ISO/IEC TR 5469:2024, key terms and definitions are primarily drawn from ISO/IEC 22989:2022, which provides fundamental concepts and terminology for artificial intelligence. This ensures a consistent understanding of AI-specific terms across the document. Additionally, the Technical Report incorporates essential definitions related to functional safety from IEC 61508-4:2010, a foundational standard in the field of functional safety.

Specifically, the document defines 'safety' as the freedom from risk which is not tolerable, aligning with established safety engineering principles. 'Functional safety' is delineated as the part of overall safety pertaining to the Equipment Under Control (EUC) and its control system, dependent on the correct functioning of Electrical/Electronic/Programmable Electronic (E/E/PE) safety-related systems and other risk reduction measures. Furthermore, 'risk' in the context of functional safety is defined as the combination of the probability of occurrence of harm and the severity of that harm. These definitions are crucial for establishing a common understanding of safety parameters and risk assessment within AI-enabled systems, allowing for a harmonized approach to their design, development, and deployment.

Governance and Institutional Framework

ISO/IEC TR 5469:2024 was developed under the auspices of ISO/IEC JTC 1/SC 42, the international standards committee specifically responsible for standardization in the area of Artificial Intelligence. This joint technical committee, established by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC), serves as the global focal point for AI standardization, taking a holistic approach that considers the entire AI ecosystem. SC 42's mandate includes addressing technology capabilities, non-technical requirements such as business, regulatory, and policy considerations, application domain needs, and ethical and societal concerns related to AI.

The development of this Technical Report was a collaborative effort, specifically involving a joint working group that combined the extensive AI expertise of ISO/IEC JTC 1/SC 42 with the deep functional safety knowledge of IEC TC 65/SC 65A. IEC TC 65/SC 65A is the committee responsible for the widely recognized IEC 61508 functional safety series of standards. This inter-committee collaboration was essential to bridge the distinct domains of AI and functional safety, ensuring that the guidance provided in ISO/IEC TR 5469:2024 is both technically sound in AI and rigorously compliant with established safety principles. SC 42 also plays a role in providing guidance to other ISO and IEC committees that are developing AI applications in their specific sectors.

Key Provisions

ISO/IEC TR 5469:2024 outlines critical considerations for integrating AI into safety-related systems, focusing on three primary areas. Firstly, it addresses the 'use of AI inside a safety-related function to realize the functionality,' providing guidance on how AI components can be directly embedded within functions that are essential for maintaining a safe state. Secondly, the report covers the 'use of non-AI safety-related functions to ensure safety for an AI-controlled equipment,' detailing how conventional safety mechanisms can be employed to supervise or safeguard equipment that is primarily controlled by AI systems. Thirdly, it explores the 'use of AI systems to design and develop safety-related functions,' acknowledging AI's potential in accelerating and optimizing the engineering processes for safety-critical systems.

The Technical Report delves into various aspects crucial for achieving functional safety with AI. Its content includes an overview of functional safety principles, a detailed examination of AI technology elements and the three-stage realization principle (data acquisition, input tasks, and processing), and a thorough analysis of the properties and related risk factors inherent in AI systems. Furthermore, it provides insights into effective verification and validation techniques tailored for AI in safety contexts, proposes control and mitigation measures to address identified risks, and outlines relevant processes and methodologies. The document also introduces 10 key 'pillars of assurance' designed to underpin a robust assurance case for AI systems in safety-critical applications, ensuring a comprehensive approach to their development and deployment.

Scope and Application

The scope of ISO/IEC TR 5469:2024 is broad and intentionally agnostic to specific industries, making its guidance applicable across a diverse range of sectors where AI systems interact with functional safety. It is designed for any organization involved in developing, providing, or using AI-based products or services within safety-related contexts. This includes, but is not limited to, critical domains such as automotive (e.g., autonomous driving systems, object detection), healthcare (e.g., medical devices, diagnostic systems), industrial automation (e.g., predictive maintenance, safety interlocks), energy (e.g., smart grid management), aviation (e.g., aircraft control systems), and manufacturing. The report's principles are relevant wherever AI technologies could potentially impact the safety of operations, equipment, or human well-being.

By offering a framework that transcends sector-specific regulations, the Technical Report aims to provide a harmonized understanding of functional safety for AI systems globally. It recognizes that while specific industries may have their own detailed safety standards (e.g., ISO 26262 for automotive, IEC 61511 for process industry), the fundamental challenges and approaches to ensuring AI safety share common ground. The document's comprehensive nature ensures that it can guide organizations of varying sizes and complexities in integrating AI responsibly, helping them navigate the complexities of AI's unpredictable nature and data-driven characteristics within safety-critical environments. This universal applicability is vital for fostering consistent safety practices across the rapidly expanding landscape of AI deployment.

Implementation Framework

The implementation framework suggested by ISO/IEC TR 5469:2024 emphasizes the integration of AI system lifecycles with established functional safety lifecycles. It explicitly references ISO/IEC 5338 (AI system life cycle processes) for managing the AI system's development, deployment, and retirement, while aligning it with the rigorous safety lifecycle defined in IEC 61508. This dual-lifecycle approach ensures that AI systems are not only technically proficient but also systematically managed for safety throughout their entire operational existence. The report highlights that within this integrated lifecycle, the presence of an Artificial Intelligence Management System (AIMS), as specified by ISO/IEC 42001:2023, is essential. An AIMS provides the organizational structure, policies, and processes necessary to establish, implement, maintain, and continually improve the responsible development and use of AI systems.

Furthermore, the Technical Report details practical implementation aspects, including verification and validation techniques specifically adapted for AI systems. It acknowledges that traditional software testing methods may not fully capture the unpredictable nature of AI, thus necessitating modified or new approaches. The document also provides guidance on control and mitigation measures, offering solutions such as designing backup functions with non-AI techniques, employing supervisor functions with constraints, or utilizing Machine Learning (ML) redundancy with output voters to enhance safety. These provisions are designed to help organizations systematically identify, assess, and treat risks associated with AI in safety-critical applications, ensuring that the system operates within tolerable risk levels.

Monitoring and Evaluation

While ISO/IEC TR 5469:2024 is a technical report providing guidance rather than prescriptive requirements, its underlying principles strongly advocate for continuous monitoring and evaluation of AI systems in functional safety contexts. The very nature of AI, particularly machine learning models that can adapt and evolve, necessitates ongoing oversight to ensure that safety properties are maintained throughout the system's operational life. The report implicitly supports the establishment of mechanisms for tracking AI system performance, detecting anomalies, and assessing potential deviations from intended safe behavior. This continuous vigilance is crucial for identifying emergent risks that may not have been apparent during initial development and testing, especially given the data-driven and sometimes opaque nature of complex AI algorithms.

The guidance on verification and validation techniques within the report, coupled with its focus on risk factors and mitigation measures, inherently promotes a dynamic approach to safety assurance. It encourages organizations to implement robust processes for collecting operational data, analyzing system outputs, and periodically re-evaluating the AI's impact on functional safety. Such evaluation should consider not only technical performance but also the effectiveness of control and mitigation strategies. The insights gained from continuous monitoring and evaluation are vital for informing iterative improvements, updating risk assessments, and ensuring that AI systems continue to meet their safety objectives in evolving operational environments. This proactive stance is fundamental to maintaining trust and ensuring the long-term responsible deployment of AI in safety-critical applications.

Relationship to Other Instruments

ISO/IEC TR 5469:2024 is strategically positioned within the broader ecosystem of international standards, acting as a crucial link between AI technologies and established functional safety practices. It explicitly references and builds upon the foundational IEC 61508 series, which is the overarching international standard for functional safety of electrical, electronic, and programmable electronic safety-related systems. The Technical Report is designed to complement existing sector-specific functional safety standards, such as ISO 26262 for automotive, IEC 62061 for machinery, and ISO 13849, by providing guidance on how AI can be safely integrated into systems governed by these standards.

Furthermore, the document integrates with other key AI standards developed by ISO/IEC JTC 1/SC 42. It draws its core terminology from ISO/IEC 22989:2022 (Artificial intelligence concepts and terminology) and relates the AI system lifecycle to ISO/IEC 5338 (AI system life cycle processes). The report also underscores the necessity of an Artificial Intelligence Management System (AIMS), as defined in ISO/IEC 42001:2023, to provide the overarching governance framework for managing AI risks and opportunities. Looking ahead, ISO/IEC TR 5469:2024 is intended to serve as the foundational document for a forthcoming technical specification, ISO/IEC TS 22440, which will provide more detailed guidance and requirements for functional safety and AI systems, indicating a progressive development in this critical area of standardization.

International Alignment

ISO/IEC TR 5469:2024 significantly contributes to international alignment in the governance of AI, particularly concerning its application in safety-critical systems. Developed by ISO and IEC, two leading global standards bodies, the report inherently promotes a common understanding and harmonized approach across national borders. The collaborative effort between ISO/IEC JTC 1/SC 42 (Artificial Intelligence) and IEC TC 65/SC 65A (Functional Safety) ensures that the guidance is globally relevant and technically robust, reflecting a consensus among international experts. This alignment is crucial for facilitating cross-border trade, enabling interoperability of AI systems, and reducing fragmentation in regulatory approaches to AI safety.

By providing a unified framework, the Technical Report assists diverse stakeholders—including developers, regulators, and users—in applying consistent safety principles to AI technologies, regardless of their geographic location or specific application domain. This fosters greater trust in AI systems on a global scale, which is essential for unlocking the full benefits of AI while mitigating its inherent risks. The report's emphasis on responsible AI adoption in functional safety environments supports broader international efforts to develop trustworthy AI, ensuring that technological advancements are accompanied by robust safety and ethical considerations that resonate worldwide.

Implementation Timeline

Adoption and Endorsement

Sources and References