Accuracy

Official/legal/standards definition: Accuracy is used in AI governance and standards to describe the closeness of an AI system’s results — observations, computations, or estimates — to the true values or values accepted as true. The ISO/IEC trustworthiness vocabulary defines accuracy as a “measure of closeness of results of observations, computations, or estimates to the true values or the values accepted as being true.”

([standards.iteh.ai](https://standards.iteh.ai/catalog/standards/iso/5ed3fe91-0264-4570-affe-87ee7bbab747/iso-iec-ts-5723-2022?utm_source=openai))

The European Artificial Intelligence Act (Regulation (EU) 2024/1689) does not provide a stand‑alone definitional clause for “accuracy,” but makes accuracy an explicit, mandatory performance requirement for high‑risk AI systems, directing providers to ensure an appropriate level of accuracy and to declare the relevant accuracy metrics in instructions for use. The Act therefore treats accuracy as a verifiable, lifecycle requirement that must be measured, documented and communicated to deployers and users.

([eur-lex.europa.eu](https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng))

Standards and risk frameworks operationalize the ISO/IEC vocabulary: the NIST AI Risk Management Framework (AI RMF 1.0) cites the ISO trustworthiness vocabulary when discussing accuracy and recommends that accuracy measurements consider computational metrics (for example false positive/false negative rates), human–AI teaming, and external validity (generalizability beyond training conditions). The OECD catalogue of metrics likewise treats accuracy as a technical metric and gives the common classification formula (accuracy = (TP + TN) / (TP + TN + FP + FN)) while noting the need to pair such measures with representative test sets and disaggregated analysis.

([airc.nist.gov](https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/?utm_source=openai))

Jurisdictional variations

• European Union (EU): The EU AI Act makes accuracy a mandatory performance requirement for high‑risk systems and requires that the levels of accuracy and relevant accuracy metrics be declared in the instructions for use; the focus is on measurable, lifecycle performance and on documenting known circumstances that affect expected accuracy. Compliance and conformity assessment regimes under the Act rely on declared metrics and harmonised standards. ([eur-lex.europa.eu](https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng))
• United States (federal): U.S. federal policy frameworks (NIST AI RMF and the White House Executive Order on Safe, Secure, and Trustworthy AI) treat accuracy as one characteristic of trustworthy AI but do not codify a single statutory definition; instead they defer to international standards (ISO) for terminology and emphasize empirical validation, testing, red‑teaming, and documentation. The Federal Trade Commission (FTC) enforces truthful, substantiated claims about AI performance and expects businesses to have empirical evidence for accuracy claims. ([airc.nist.gov](https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/?utm_source=openai))
• U.S. states (example—Colorado & California): State laws that regulate AI in particular contexts focus on ensuring accurate, non‑discriminatory outcomes rather than defining the term technically. Colorado’s high‑risk AI law requires impact assessments, annual reviews, consumer notices and opportunities to correct incorrect personal data — obligations that functionally require measuring accuracy, documenting error modes, and providing remediation processes. California’s transparency and sector rules (e.g., generative AI producer transparency, health‑care AI notices) similarly push producers to disclose limitations and enable verification. ([leg.colorado.gov](https://leg.colorado.gov/bills/sb24-205))
• International (standards & multilateral frameworks): ISO/IEC standards (notably ISO/IEC TS 5723 and ISO/IEC 22989) provide the technical vocabulary and allow jurisdictions and frameworks (NIST, OECD, UNESCO) to adopt a common meaning for accuracy: a measure of closeness to true/accepted values. OECD’s metrics catalogue gives practical metric definitions and recommended test practices; UNESCO frames accuracy as part of trustworthiness and ethical deployment (accuracy supports respect for rights and avoidance of harms). ([standards.iteh.ai](https://standards.iteh.ai/catalog/standards/iso/5ed3fe91-0264-4570-affe-87ee7bbab747/iso-iec-ts-5723-2022?utm_source=openai))

Context, scope and practical implications for businesses

In regulatory and standards contexts, accuracy is not only a statistical spot‑measurement; it must be operationalized in relation to the AI system’s intended purpose, applicable populations, and reasonably foreseeable uses and misuses. Practically this means businesses must: (a) select appropriate accuracy metrics (overall accuracy, class‑specific accuracy, precision/recall, AUC, calibration), (b) define and publish test sets representative of expected use, (c) disaggregate performance across relevant subgroups, (d) declare the accuracy level(s) in product documentation and instructions for use (where required, e.g., by the EU AI Act), and (e) implement monitoring, re‑validation and update processes across the lifecycle to detect performance drift. Failure to substantiate claimed accuracy can trigger enforcement (consumer protection, misleading claims) and, in high‑risk contexts, may undermine conformity assessments and market access. ([eur-lex.europa.eu](https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng))

Key requirements or criteria (typical across frameworks)

• Use a clear, purpose‑bound metric set (e.g., TP/TN/FP/FN derived metrics; calibration; task‑specific scores). ([oecd.ai](https://oecd.ai/en/catalogue/metrics/accuracy?utm_source=openai))
• Evaluate on representative, well‑documented test sets that reflect expected operational conditions and report external validity/generalizability. ([airc.nist.gov](https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/?utm_source=openai))
• Document accuracy levels and measurement methodology in technical documentation and user instructions (EU high‑risk requirement). ([eur-lex.europa.eu](https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng))
• Disaggregate performance to detect group‑differential accuracy and examine fairness trade‑offs. ([airc.nist.gov](https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/?utm_source=openai))
• Continuously monitor performance in production, address drift, and incorporate remediation and human oversight controls. ([eur-lex.europa.eu](https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng))

Examples

1) A medical image classifier: reporting a single overall accuracy is insufficient — regulators and clinicians expect sensitivity/recall (false negatives), specificity (false positives), calibration (predicted probability vs observed outcomes), subgroup performance (age, sex, ethnicity), and documentation of test cohorts. 2) A resume‑screening model: high overall accuracy that masks much lower accuracy for a protected subgroup creates legal and reputational risk even if the mean accuracy looks acceptable. In both cases, the EU AI Act’s requirement to declare accuracy metrics and NIST’s call to pair metrics with representative test sets illustrate how measurement, documentation, and context combine into regulatory compliance. ([eur-lex.europa.eu](https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng))

Cross‑references: see related concepts: robustness (resilience to changed conditions), validation (evidence that requirements for intended use are met), precision/recall and false‑positive/false‑negative rates, calibration, and human oversight. Standards and frameworks treat these concepts as complementary: accuracy measures performance under defined conditions, while robustness/validation/monitoring address behavior across the lifecycle and under stress or distributional shift. ([standards.iteh.ai](https://standards.iteh.ai/catalog/standards/iso/5ed3fe91-0264-4570-affe-87ee7bbab747/iso-iec-ts-5723-2022?utm_source=openai))