Data Poisoning
Attack on AI systems by manipulating training data to cause incorrect or harmful model behavior.
How the laws define this
Both regulatory sources agree that data poisoning is an intentional adversarial attack where corrupted, misleading, or manipulated data is introduced into an AI model's training dataset to force it to learn incorrect patterns or associations, ultimately leading to security vulnerabilities or degraded, unintended performance. The key divergence lies in the scope of data covered, the specific harms emphasized, and the inclusion of regulatory compliance controls. The Artificial Intelligence Playbook for the UK Government takes a broader operational view, explicitly extending the definition to include validation data, pointing to specific adversarial behaviours, and outlining concrete preventative measures such as supplier due diligence, provenance checks, and validation or testing controls. In contrast, 人工智能拟人化互动服务管理暂行办法（征求意见稿） confines its definition strictly to the training dataset, highlighting biased outputs as a distinct potential risk of such corruption while omitting explicit management actions or validation controls within its text.
Synthesised from the 2 statutory definitions below. Each one is quoted in full, with its source.
How jurisdictions differ
What is distinctive in each definition, binding law in force first.
| Jurisdiction | Instrument | Status | What is distinctive |
|---|---|---|---|
| China | Interim Measures for the Administration of Personified Interactive Services Powered by Artificial Intelligence | In Force | Emphasizes security vulnerabilities and biased outputs as consequences, linking to national security in anthropomorphic AI services. |
| United Kingdom | Artificial Intelligence Playbook for the UK Government | In Force (Amended) | Highlights supplier due diligence, provenance checks, and validation/testing as mitigation controls in government AI use. |
What it triggers
Definitions (2)
The intentional introduction or manipulation of training or validation data to cause a model to learn incorrect associations or vulnerabilities, degrading performance or enabling specific adversarial behaviours. The Playbook addresses data poisoning through supplier due diligence, provenance checks, and validation/testing controls.
A type of attack where an adversary injects corrupted or misleading data into an AI model's training dataset, causing the model to learn incorrect patterns or behave in unintended ways, potentially leading to security vulnerabilities or biased outputs.
Not to be confused with
Data governance refers to policies and processes for managing data throughout the AI lifecycle, while data poisoning is a specific malicious attack on training data.
Cybersecurity requirements are measures to defend against threats like data poisoning, not the attack itself.
Questions and answers
What measures does the UK AI Playbook suggest against data poisoning?
It recommends supplier due diligence, provenance checks, and validation and testing controls. These help ensure model integrity in government applications.
How does China's regulation address data poisoning in AI?
It defines it in the context of anthropomorphic AI services, focusing on preventing security vulnerabilities and biased outputs through regulatory oversight.
What are practical steps to mitigate data poisoning in AI systems?
Use provenance checks, validation testing, and supplier vetting as described in the UK guidance.
Related Terms
Training Data
Data used to develop and train AI models to perform their intended functions....
Data Governance
Policies, roles and processes that ensure data used across the AI lifecycle is fit for purpose, traceable, secure and managed in line with legal and operational requirements....
Cybersecurity Requirements
Technical and organisational measures that ensure AI systems resist, detect, respond to, and recover from cyber threats across their lifecycle....