Algorithmic Discrimination
Unfair or unlawful differential treatment or impact caused by an AI or algorithmic system against individuals or groups, often on protected characteristics.
How the laws define this
Regulators uniformly anchor algorithmic discrimination in the adverse, differential treatment or disparate impact experienced by individuals or groups based on protected characteristics resulting from the design, development, or deployment of automated systems. The central point of divergence lies in whether the standard requires a strictly illegal act or extends to broader notions of unfairness. Colorado SB24-205, Connecticut SB 2, Virginia Commission on Artificial Intelligence, and Guidance on Algorithmic Discrimination and the New Jersey Law Against Discrimination explicitly limit the concept to unlawful treatment or impact. In contrast, House Bill No. 7913 (Artificial Intelligence Regulation Act - AI Bill of Rights), the FTC Policy Statement on AI and Section 5 of the FTC Act, and 最高人民法院关于依法审理涉人工智能纠纷案件的意见 apply a broader scope covering outcomes that are unfair, biased, or prejudicial without requiring explicit illegality. Additionally, Concerning Consumer Protections in Interactions with Artificial Intelligence Systems clarifies that using AI to expand diversity or remedy historical discrimination is not algorithmic discrimination.
Synthesised from the 13 statutory definitions below. Each one is quoted in full, with its source.
How jurisdictions differ
What is distinctive in each definition, binding law in force first. Showing 10 of the 13 instruments that define this term; all are listed under Definitions below.
| Jurisdiction | Instrument | Status | What is distinctive |
|---|---|---|---|
| Colorado, United States | Colorado SB24-205 — Consumer Protections for Artificial Intelligence Act | Repealed | Expands protected characteristics to include limited English proficiency and veteran status, reproductive health, and genetic information beyond typical lists. |
| Virginia, United States | Virginia Commission on Artificial Intelligence (established by SB 487, 2024 Session) | In Force | Limits to unlawful discrimination resulting from AI systems use, without specifying protected characteristics or disparate impact. |
| China | Opinions of the Supreme People’s Court on the Trial of Cases Involving Artificial Intelligence Disputes | In Force | Emphasizes judicial prevention and elimination of biases to ensure fairness and equal treatment in automated decision-making processes. |
| New Jersey, United States | Guidance on Algorithmic Discrimination and the New Jersey Law Against Discrimination | In Force | Ties discrimination specifically to design, training, or deployment of automated decision-making tools under the NJLAD. |
| New York, United States | An act to amend the civil rights law and the executive law, in relation to the use of artificial intelligence systems | Under Review | Explicitly covers discrimination from design, development, or use of AI systems, listing sexual orientation among protected characteristics. |
| Connecticut, United States | An Act Concerning Artificial Intelligence (Connecticut SB 2) | Withdrawn | Includes domestic violence victim status and marital status, and gender identity or expression among protected characteristics. |
| Washington, United States | Concerning algorithmic discrimination | Withdrawn | Broadens to unfair, unjust, or prejudicial outcomes from biased data, flawed model design, or inappropriate application, not limited to protected characteristics. |
| United Kingdom | Artificial Intelligence (Regulation and Workers' Rights) Bill | Withdrawn | Limits to biases in AI systems used for employment decisions like recruitment and performance in HR functions. |
| Philippines | House Bill No. 7913 (Artificial Intelligence Regulation Act - AI Bill of Rights) | Withdrawn | Focuses on systematic disadvantage from biased data, design, or deployment, underpinning obligations for fairness assessments and bias testing. |
| Colorado, United States | Increase Transparency for Algorithmic Systems | In Force | Plainly limits to unfair, unlawful, or biased treatment by algorithmic decision systems based on protected characteristics. |
What it triggers
- In Colorado, developers and deployers of high-risk AI systems must mitigate algorithmic discrimination and conduct impact assessments.source
- The Guidance on Algorithmic Discrimination sets out how New Jersey anti-discrimination law applies to entities using automated decision-making systems.source
- Under Increase Transparency for Algorithmic Systems, the Act aims to prevent outcomes where algorithmic decision systems result in unfair, unlawful, or biased treatment.source
Definitions (13)
Any condition in which an AI system's use results in unlawful differential treatment or a disparate impact disfavoring individuals based on actual or perceived protected characteristics (e.g., age, race, disability, national origin, sex, religion, genetic information, limited English proficiency, veteran status, reproductive health). The term anchors the law's central non-discrimination objective and enforcement focus.
Discrimination that results from the design, training, or deployment of automated decision-making tools and leads to unlawful differential treatment or disparate impacts against protected characteristics under the NJLAD.
AI-generated outcomes that systematically disadvantage, exclude, or treat individuals unfairly based on protected characteristics, resulting from biased data, design, or deployment practices. The definition underpins obligations for fairness assessments, bias testing, mitigation, and remediation to protect non-discrimination rights.
Any condition where the use of an artificial intelligence system results in an unlawful differential treatment or impact that disfavors an individual or group of individuals on the basis of actual or perceived age, color, disability, domestic violence victim status, gender identity or expression, marital status, national origin, race, religion, sex, sexual orientation, or veteran status.
Unlawful discrimination against individuals or groups resulting from the use of artificial intelligence systems, a key area the Commission is tasked to study and prevent.
Algorithmic discrimination refers to the differential treatment or impact on individuals or groups that arises from the design, development, or deployment of algorithms and automated systems, leading to outcomes that are unfair, unjust, or prejudicial. This can occur due to biased data, flawed model design, or inappropriate application, often exacerbating existing societal inequalities.
Algorithmic discrimination refers to situations where AI systems produce outcomes that unfairly disadvantage individuals or groups based on protected characteristics, such as race, gender, or age, in areas like credit, employment, or housing.
Any differential treatment or impact on individuals or groups based on protected characteristics (such as race, gender, religion, national origin, age, disability, sexual orientation, etc.) that results from the design, development, or use of an artificial intelligence system.
Outcomes where an algorithmic decision system results in unfair, unlawful, or biased treatment of individuals or groups based on protected characteristics, which the Act aims to prevent.
Discrimination that arises from biases embedded in artificial intelligence systems used for employment decisions, leading to unfair or prejudiced outcomes against individuals or groups based on protected characteristics. The Bill aimed to prevent such biases in recruitment, performance, and other HR functions.
Defined as any condition where the use of an AI system results in unlawful differential treatment or impact based on protected classes under Colorado and federal law, such as race, disability, age, or gender. The act clarifies that using AI to expand diversity or remedy historical discrimination is not considered algorithmic discrimination.
The adverse differential treatment of individuals or groups based on their actual or perceived characteristics by a high-risk artificial intelligence system, which the bill aimed to mitigate.
Algorithmic discrimination occurs when AI systems produce results that unfairly disadvantage certain groups or individuals based on protected characteristics. The Opinions emphasize the need to prevent and eliminate such biases, ensuring fairness and equal treatment in automated decision-making processes.
Not to be confused with
Algorithmic bias refers to systematic errors in AI outputs causing unfair outcomes, while algorithmic discrimination is the resulting unfair differential treatment or impact on protected groups.
Disparate impact is the discriminatory effect disproportionately affecting protected groups without intent, whereas algorithmic discrimination encompasses both differential treatment and disparate impact.
Questions and answers
What protected characteristics are covered by algorithmic discrimination in Colorado SB24-205?
It covers age, race, disability, national origin, sex, religion, genetic information, limited English proficiency, veteran status, and reproductive health. Using AI to expand diversity or remedy historical discrimination is not considered algorithmic discrimination.
Does algorithmic discrimination include outcomes from AI design flaws?
Yes, definitions like those in New York S1169 and Washington HB 1954 explicitly include differential treatment or impact resulting from the design, development, or deployment of AI systems.
What obligations arise from algorithmic discrimination in high-risk AI systems?
Colorado SB24-205 required mitigation and impact assessments for high-risk systems. New Jersey Guidance clarifies application of anti-discrimination law to automated tools.
Related Terms
Algorithmic Bias
Systematic errors in AI outputs causing unfair outcomes for groups....
Disparate Impact
Discriminatory effect of AI systems that disproportionately affects protected groups, even without discriminatory intent....
Non-Discrimination
Requirement that AI systems do not produce unjustified differential treatment or disproportionate adverse impacts against individuals or groups based on protected or analogous characteristics....
Fairness
Absence of unjustified bias or discrimination in AI outputs and equitable treatment and access across individuals and groups....
Impact Assessment
A systematic, documented review (ex‑ante or ongoing) of how an AI system may affect individuals, groups, organisations or society and measures to mitigate harms....