Guardrails
Technical mechanisms and constraints built into AI systems to prevent harmful outputs or behaviors.
How the laws define this
Both instruments agree that guardrails consist of operational safeguards and risk controls—including risk assessment frameworks, model testing, and supplier assurance—designed to prevent or limit harms arising from systems. They diverge primarily in their regulatory weight, operational context, and mandatory scope. The Proposals Paper for Introducing Mandatory Guardrails for AI in High‑Risk Settings establishes guardrails as ten proposed mandatory requirements that apply comprehensively across the entire AI lifecycle and supply chain for developers and deployers in high-risk settings, covering explicit obligations such as accountable officials, record-keeping, conformity assessments, and transparent reporting. In contrast, the Public Service AI Framework treats guardrails as operational safeguards meant specifically for public service use, emphasizing controls such as security- and privacy-by-design. While the Proposals Paper for Introducing Mandatory Guardrails for AI in High‑Risk Settings creates a binding, ten-part compliance standard across the supply chain for high-risk applications, the Public Service AI Framework focuses on functional safeguards tailored to government and public service environments.
Synthesised from the 2 statutory definitions below. Each one is quoted in full, with its source.
What it triggers
- New Zealand Public Service AI Framework recommends operational safeguards including risk assessment frameworks, model validation and testing, supplier assurance, and security- and privacy-by-design controls to prevent or limit harms from AI systems in public service use.source
- Proposals Paper for Introducing Mandatory Guardrails for AI in High-Risk Settings would require developers and deployers of high-risk AI to implement ten specified requirements across the AI lifecycle and supply chain.
Definitions (2)
The set of ten proposed mandatory requirements applying across the AI lifecycle and supply chain (accountable official, lifecycle risk management, data governance and provenance, pre‑deployment and adversarial testing, transparent reporting and disclosures, meaningful human oversight, ongoing monitoring and incident reporting, supply‑chain transparency, record‑keeping/documentation, and conformity assessment) that developers and deployers must implement for high‑risk AI.
Operational safeguards including risk assessment frameworks, model validation and testing, supplier assurance, and security‑ and privacy‑by‑design controls intended to prevent or limit harms from AI systems in public service use.
Not to be confused with
AI Safety focuses on measures to prevent unreasonable harm during intended use or misuse, while guardrails are specific mechanisms or requirements to constrain outputs or behaviors.
Red Teaming is a testing process to find vulnerabilities, whereas guardrails are the built-in constraints or requirements that address those vulnerabilities.
Questions and answers
What safeguards does the New Zealand Public Service AI Framework recommend for guardrails?
It recommends operational safeguards including risk assessment frameworks, model validation and testing, supplier assurance, and security- and privacy-by-design controls.
What are the ten requirements in the Proposals Paper for guardrails?
The ten proposed mandatory requirements are accountable official, lifecycle risk management, data governance and provenance, pre-deployment and adversarial testing, transparent reporting and disclosures, meaningful human oversight, ongoing monitoring and incident reporting, supply-chain transparency, record-keeping/documentation, and conformity assessment.
How do guardrails differ from AI safety?
Guardrails refer to specific technical mechanisms, safeguards, or requirements to prevent harmful outputs, while AI safety encompasses broader measures to ensure no unreasonable harm during use or misuse.
Related Terms
AI Safety
Measures and engineering practices ensuring AI systems do not cause unreasonable harm to people, property, or the environment during intended use or foreseeable misuse....
Alignment
Ensuring an AI system’s objectives, outputs and behaviour are consistent with specified human goals, values, laws and expectations....
Red Teaming
A structured, adversarial testing process that simulates intentional misuse to find vulnerabilities, failure modes, and harms in AI systems before deployment....