Hallucination
When an AI (often a generative model) produces fluent but factually incorrect, fabricated, or unsupported outputs presented as if true.
How the laws define this
Regulators uniformly define hallucinations as generative model outputs that seem plausible, coherent, or convincing on the surface but are factually incorrect, false, fabricated, or unsupported by evidence or training data. All frameworks treat this phenomenon as a core safety, reliability, or operational risk requiring active measurement, testing, and mitigation. Crucially, the instruments diverge on what specific output characteristics qualify. The Artificial Intelligence Playbook for the UK Government, the Generative AI Ethics Guidebook (NIA / KCC – sectoral guide for generative AI), and the Ethics Guide for Generative Artificial Intelligence Use in Scientific Research and Publication in Higher Education Institutions strictly focus on outputs that appear plausible, coherent, or credible. By contrast, the Model AI Governance Framework for Generative AI expands hallucinations to also include outputs that are nonsensical or unfaithful to the input prompt.
Synthesised from the 5 statutory definitions below. Each one is quoted in full, with its source.
How jurisdictions differ
What is distinctive in each definition, binding law in force first. Showing 3 of the 5 instruments that define this term; all are listed under Definitions below.
| Jurisdiction | Instrument | Status | What is distinctive |
|---|---|---|---|
| South Korea | Generative AI Ethics Guidebook (NIA / KCC – sectoral guide for generative AI) | In Force | Emphasizes outputs unsupported by training data and frames hallucination as a key risk to monitor, test for, and mitigate. |
| United Kingdom | Artificial Intelligence Playbook for the UK Government | In Force (Amended) | Adds operational risk framing with specific mitigation via testing, evaluation, and user-facing disclosures in government context. |
| Turkey | Ethics Guide for Generative Artificial Intelligence Use in Scientific Research and Publication in Higher Education Institutions | In Force | Limits to scientific research context, highlighting invented citations or falsified data as examples, and advises detection and correction in academic work. |
What it triggers
- In South Korea, the Generative AI Ethics Guidebook treats hallucinations as a key risk to monitor, test for, and mitigate.source
- Under the UK Artificial Intelligence Playbook for the UK Government, hallucinations should be identified, measured, and mitigated through testing, evaluation, and user-facing disclosures.source
- In Turkey, the Ethics Guide for Generative Artificial Intelligence Use in Scientific Research and Publication advises researchers to detect, correct, and not rely on hallucinated content in scientific work.source
Definitions (5)
Outputs produced by a generative model that are plausible in form but incorrect, fabricated, or unsupported by evidence; identified as a key safety risk to be measured, tested, and mitigated through evaluation and alignment work.
A phenomenon where a generative model produces outputs that are coherent or convincing but factually incorrect or unsupported by training data; the Guidebook treats this as a key risk to monitor, test for, and mitigate.
Model outputs that appear coherent or credible but are factually incorrect, fabricated, or unsupported by evidence—examples include invented citations or falsified data. The Guide requires researchers to detect, correct, and not rely on hallucinated content in scientific work.
A model-generated output that appears coherent or plausible yet is false, unfounded, or not supported by the model's training data or factual sources. The Playbook treats hallucinations as an operational risk to be identified, measured, and mitigated through testing, evaluation and user-facing disclosures.
Instances where a generative AI model produces outputs that are factually incorrect, nonsensical, or unfaithful to the input prompt while appearing plausible, representing a key safety and reliability concern addressed in the framework.
Not to be confused with
Confabulation refers specifically to instances of generating factually incorrect, nonsensical, or inconsistent outputs presented as truthful, while hallucination more broadly covers plausible but unsupported model outputs.
Questions and answers
What obligations does the South Korean Generative AI Ethics Guidebook impose regarding hallucinations?
It treats hallucinations as a key risk and calls for monitoring, testing, and mitigation by AI developers, operators, and users.
How does the UK AI Playbook address hallucinations in government use?
It identifies hallucinations as an operational risk and sets out measures like testing, evaluation, and user-facing disclosures to mitigate them.
What does the Turkish Ethics Guide say about using hallucinated content in research?
It advises researchers to detect and correct hallucinated content, such as invented citations or falsified data, and not to rely on it in scientific work.
Related Terms
Confabulation
Instances where GAI systems generate factually incorrect, nonsensical, or inconsistent outputs presented as truthful....
Generative AI
A class of AI systems that produce new digital content (text, image, audio, video, code) by modelling and emulating patterns in training data....
Data Quality
The fitness of dataset(s) for an AI system’s intended purpose, judged by attributes like relevance, accuracy, representativeness, completeness and provenance....