UK AI Safety Standards for Education

Overview

The document titled Generative AI: product safety standards issued by the Department for Education provides guidance for the development, deployment, and use of generative artificial intelligence (AI) products and systems within educational settings across England. Primarily targeted at edtech developers and suppliers, the guidance also assists schools and colleges in evaluating and procuring AI products that meet safety criteria. The core objective of these standards is to ensure that generative AI technologies integrated into learning environments are safe, reliable, and do not pose physical or psychological hazards to users. First published on 22 January 2025 as Generative AI: product safety expectations, the document was revised and retitled on 19 January 2026.

Definitions

The "Generative AI: product safety standards" document, while not providing a formal glossary, implicitly defines several key terms through its application and context. A central concept is "Generative AI products," which refers to artificial intelligence systems capable of creating new content, such as text, images, or other media, in response to user prompts. Within the scope of this guidance, these products are specifically those "deployed, marketed, or made accessible for use in educational settings," encompassing schools and colleges in England. The document distinguishes between "learner-facing products," which directly interact with students (e.g., personalised learning tools, digital assistants, research aids, engagement platforms), and "teacher-facing products," which support educators and administrators (e.g., assessment tools, administrative management systems).

Crucial to the safety objectives are definitions related to content and user interaction. "Harmful or inappropriate content" refers to material that users should be prevented from generating or accessing, with filtering mechanisms expected to be adjusted based on "different levels of risk, age, appropriateness and the user’s needs," including those with Special Educational Needs and Disabilities (SEND). The term "cognitive offloading" is introduced in the context of monitoring, referring to instances where AI might excessively perform cognitive tasks for the user, potentially hindering their own cognitive development. The guidance also touches upon "jailbreaking," which implies attempts by users to circumvent the safety and security features of an AI product to access prohibited material or reprogram functionalities, necessitating robust protection measures against such actions. These contextual definitions are vital for developers and educators to understand the scope and intent of the safety standards.

Governance and Institutional Framework

The "Generative AI: product safety standards" document operates within the broader regulatory landscape of the United Kingdom, specifically applying to England. While the document itself is guidance rather than a statutory instrument, it is published by the UK government, implicitly under the purview of the Department for Education (DfE), which is responsible for education and children's services. The guidance explicitly states that adhering to these standards can help schools, colleges, edtech developers, and suppliers comply with a range of existing, legally binding regulations and frameworks. This establishes a robust institutional framework where the DfE's guidance acts as a practical interpretation and application of higher-level legal obligations in the context of generative AI.

The document directly references several key pieces of legislation and guidance that form its foundational governance framework. These include the Keeping children safe in education (particularly parts 1, 2, and 5), which sets out statutory guidance for schools and colleges on safeguarding. It also refers to the Filtering and monitoring standards for schools and colleges, which ensures appropriate online safety. The Public Sector Equality Duty is mentioned, guiding public authorities on their obligations to eliminate discrimination, advance equality of opportunity, and foster good relations. Furthermore, the guidance highlights the relevance of the Online Safety Act 2023 (OSA) for generative AI services that allow user-shared content or search live websites, requiring providers to tackle illegal content and protect children. Data protection is anchored in the General Data Protection Regulation (GDPR) Article 35, which mandates Data Protection Impact Assessments (DPIAs) for high-risk processing, and the Information Commissioner’s Office (ICO) age appropriate design code, specifically section 11 on monitoring. Security aspects align with the Cyber Security Standards for Schools and Colleges and the Computer Misuse Act 1990. This intricate web of references positions the generative AI safety standards as an integral part of the UK's broader regulatory ecosystem for technology and child protection.

Key Focus Areas

The "Generative AI: product safety standards" document outlines several critical focus areas to ensure the safe and responsible deployment of AI in educational settings. A primary area is the "Stated purpose" and "Educational use cases" of generative AI products. Developers and suppliers are required to clearly articulate the intended purpose of their AI tools, specifying target demographics (e.g., age, SEND status) and learning focus. The guidance categorizes intended use cases, including content creation and delivery, personalised learning and accessibility, assessment and analytics, digital assistants, research and writing aids, learner engagement, and administrative tasks. Any claims made about the product's capabilities must be supported by robust and transparent evidence, and new features or modifications necessitate a review of the stated purpose and use cases.

Another crucial focus is "Filtering," particularly for learner-facing products. The standards mandate that generative AI products must effectively and reliably prevent users from generating or accessing harmful or inappropriate content. This includes maintaining filtering standards throughout interactions, adjusting filters based on risk levels, age, appropriateness, and user needs (e.g., SEND users), and effectively moderating multimodal content across languages, images, misspellings, and abbreviations. Content moderation must also be contextually aware. "Monitoring and reporting" is another vital component, requiring robust activity logging procedures for learner-facing products. This includes recording prompts and responses, analyzing performance metrics, and alerting local supervisors to attempts or successes in accessing harmful content. Products must also alert users with age-appropriate notifications when content is blocked, identify and alert supervisors to safeguarding disclosures, maintain current safeguarding lead contact details, and generate understandable reports on prohibited content access. The guidance also specifies monitoring for cognitive offloading, personal and emotional engagement, and usage duration.

"Security" is a significant focus, applying to both learner and teacher-facing products. Products must be secured against malicious use or exposure to harm, prioritizing reliability, security, robustness, and safe operation under various conditions, including adversarial attacks. Specific expectations include robust protection against 'jailbreaking' and unauthorised modifications, the ability for administrators to set different permission levels, prompt implementation of bug fixes and updates, sufficient testing of new versions for safety compliance, and robust password protection or authentication methods. Compatibility with the Cyber Security Standards for Schools and Colleges is also required. Finally, "Privacy and data protection" is paramount, mandating compliance with relevant data protection legislation, including GDPR. Products must have a robust approach to data handling, transparency around personal data processing, and a lawful basis for data collection. This includes providing clear, comprehensive, and age-appropriate privacy notices at regular intervals, detailing data types, collection, processing, storage, sharing, and cross-border transfers. Developers are also expected to conduct Data Protection Impact Assessments (DPIAs) for high-risk data processing activities. The latest update also introduced new standards on cognitive development, emotional and social development, mental health, and manipulation, further broadening the scope of safety considerations.

Implementation Framework

The implementation framework for the "Generative AI: product safety standards" is designed to create a shared responsibility between edtech developers, suppliers, and educational institutions in England. For developers and suppliers, the framework mandates proactive engagement with the standards from the product design phase. They are expected to clearly state the intended purpose and use cases of their generative AI products, providing detailed information on target demographics and learning focus. Crucially, any claims made about a product's impact or capabilities must be substantiated by robust and transparent evidence, ensuring that schools and colleges can make informed decisions based on verified information. When new features or modifications are introduced, developers are required to review the intended purpose and update use case declarations accordingly, maintaining a dynamic approach to product safety throughout the lifecycle.

The responsibility for assuring that these standards are met is explicitly placed on the suppliers of systems and tools working directly with schools and colleges, even if certain standards need to be addressed further up the supply chain. This means that direct providers to educational institutions must ensure that the generative AI products they offer, regardless of their origin, comply with the outlined safety expectations. For schools and colleges, the guidance serves as a practical tool for assessment. It empowers them to scrutinize AI products before procurement and deployment, ensuring that the chosen technologies align with safeguarding duties, data protection obligations, and overall educational objectives. By providing a clear benchmark, the framework facilitates a consistent approach to AI safety across the education sector, promoting a culture of due diligence and responsible technology adoption.

Monitoring and Evaluation

The "Generative AI: product safety standards" places significant emphasis on robust monitoring and evaluation mechanisms, particularly for learner-facing products. The core requirement is for generative AI products to maintain comprehensive "activity logging procedures." This includes systematically recording all input prompts from users and the corresponding AI-generated responses. Beyond simple logging, products are expected to analyze performance metrics to understand how the AI is being used and to identify any deviations from safe operation. A critical aspect of monitoring is the system's ability to "identify and alert local supervisors" (such as teachers or safeguarding leads) to searches for, or access to, harmful or inappropriate content. This proactive alerting mechanism is vital for timely intervention and safeguarding.

Further to alerting supervisors, the standards require products to "alert and signpost the user to appropriate guidance and support resources" when prohibited content is attempted or accessed. This includes generating "real-time user notifications in age-appropriate language" explaining why content has been blocked. Products must also be capable of "identifying and alerting local supervisors of disclosures that indicate a possible safeguarding issue," ensuring that potential risks to children are promptly escalated. To facilitate this, systems must maintain current contact details for an institution’s Designated Safeguarding Lead (DSL) or equivalent authority, requiring input during setup, confirming details before activation, and allowing for easy updates. Finally, the guidance mandates the generation of "reports and trends on access and attempted access of prohibited content," presented in a format understandable to non-expert staff, to avoid imposing undue burdens on local supervisors. Beyond safety, products should monitor and report data on the rate and amount of "cognitive offloading," the level of personal and emotional engagement (without disclosing content), and the duration of usage by individual learners, providing valuable insights for educators.

Penalties, Liability, and Appeals

It is important to clarify that the "Generative AI: product safety standards" document itself is a piece of guidance and, as such, does not directly impose penalties, assign liability, or establish an appeals process for non-compliance with its recommendations. Its primary function is to outline best practices and expectations for generative AI products in educational settings in England. However, the guidance explicitly states that meeting these standards can help developers, suppliers, schools, and colleges comply with a range of existing, legally binding regulations and acts. Non-compliance with these underlying statutory instruments can indeed lead to significant penalties, legal liability, and established appeals processes.

For instance, generative AI services falling within the scope of the Online Safety Act 2023 are subject to its enforcement regime, which includes substantial fines for providers failing to meet their duties regarding illegal content and child protection. Similarly, breaches of the General Data Protection Regulation (GDPR), particularly concerning data protection impact assessments (Article 35) or the handling of children's data, can result in severe financial penalties imposed by the Information Commissioner's Office (ICO). Non-adherence to the Computer Misuse Act 1990, which addresses unauthorised access and modifications to computer material, carries criminal offences. While the DfE guidance itself is not enforceable in the same way, it serves as a benchmark for demonstrating due diligence and responsible practice. Failure to follow such guidance could be used as evidence of negligence or insufficient safeguarding measures in legal proceedings related to the referenced binding regulations, thereby indirectly impacting liability and potential for penalties.

Relationship to Other Instruments

The "Generative AI: product safety standards" document is meticulously designed to complement and support compliance with a range of existing legal and regulatory instruments within the UK. It explicitly states that adherence to these AI safety standards can assist schools, colleges, edtech developers, and suppliers in fulfilling their obligations under several key frameworks. For safeguarding, the guidance helps schools and colleges comply with Keeping children safe in education, particularly parts 1, 2, and 5, which emphasize preventing access to harmful content and broader safeguarding responsibilities. It also aligns with the Filtering and monitoring standards for schools and colleges, ensuring that AI products contribute to a safe online environment.

In the realm of broader legal obligations, the standards support compliance with the Public Sector Equality Duty, by promoting inclusive and accessible AI solutions. A significant connection is made to the Online Safety Act 2023 (OSA). Generative AI services that allow user content sharing or search live websites are regulated under the OSA, which mandates providers to conduct risk assessments for illegal and child-harmful content, proactively mitigate risks, and use effective age assurance. The DfE guidance provides practical steps for AI products to contribute to these OSA duties. Data protection is addressed through its compatibility with the General Data Protection Regulation (GDPR) Article 35, which requires Data Protection Impact Assessments (DPIAs) for high-risk processing, and the Information Commissioner’s Office (ICO) age appropriate design code, particularly section 11 on monitoring and transparency with children. Finally, for security, the standards are compatible with the Cyber Security Standards for Schools and Colleges and help schools and colleges comply with the Computer Misuse Act 1990, by setting expectations for robust product security and preventing unauthorised access or modification.

International Alignment

The "Generative AI: product safety standards" document is primarily focused on the domestic context of England's educational settings and does not explicitly detail its alignment with international AI regulatory frameworks or standards. The guidance is tailored to the specific legal and policy landscape of the United Kingdom, drawing heavily on existing UK legislation such as the Online Safety Act 2023, the Cyber Security Standards for Schools and Colleges, and the Computer Misuse Act 1990, as well as broader UK safeguarding and data protection guidance. While the document references the General Data Protection Regulation (GDPR), it does so in the context of UK GDPR, which, while originally derived from EU law, now operates as part of UK domestic law post-Brexit.

Given its specific focus on the UK education sector, the guidance does not outline mechanisms for cross-border cooperation, mutual recognition of standards with other nations, or engagement with international bodies like the OECD, UNESCO, or the European Union on AI regulation. Its primary objective is to provide clear, actionable safety expectations for generative AI products within its defined national scope. However, by promoting principles of safety, transparency, data protection, and risk mitigation, the UK's approach may inherently share common objectives with international efforts to regulate AI, even if direct alignment is not explicitly stated. Developers and suppliers operating internationally would still need to ensure compliance with the specific requirements of each jurisdiction in which their products are deployed.

Implementation Timeline

The guidance was first published by the Department for Education on 22 January 2025 under the title Generative AI: product safety expectations. On 19 January 2026, the guidance was substantially revised, expanded, and retitled Generative AI: product safety standards, introducing four new standard categories addressing cognitive development, emotional and social development, mental health, and manipulation.

Sources and References