Disparate Impact
Definition
Disparate impact is a legal doctrine, originating in US employment discrimination law (Griggs v. Duke Power, 1971), that prohibits practices that disproportionately harm protected groups even without discriminatory intent. Applied to AI systems, disparate impact means that an algorithm can be illegal if it produces substantially different outcomes for groups defined by race, sex, age, national origin, or disability—regardless of whether protected attributes were explicitly used as features. The '80% rule' (four-fifths rule) is a common threshold: if the selection rate for any group is less than 80% of the highest-performing group's rate, disparate impact is indicated. EEOC, CFPB, and HUD have all applied disparate impact doctrine to algorithmic decision systems.
Why It Matters
Disparate impact is one of the most significant legal risks in deploying AI for consequential decisions. Hiring algorithms, lending models, insurance pricing, and criminal justice tools have all faced disparate impact litigation and regulatory action. The key insight is that using neutral-seeming features (zip code, employment gap, educational institution) can encode protected attributes through correlation, creating disparate impact without explicit discrimination. Organizations deploying AI for hiring, lending, housing, insurance, or public services must conduct disparate impact analysis before deployment and monitor for disparate impact in production—the legal standard requires both testing and correction.
How It Works
Disparate impact analysis: (1) identify all protected groups relevant to the jurisdiction and use case (race, sex, age, disability, national origin, religion); (2) calculate the selection rate (positive outcome rate) for each group; (3) compute the adverse impact ratio: lowest group rate / highest group rate; (4) flag ratios below 0.8 (80% rule) as potential disparate impact; (5) investigate whether the disparity is justified by job-related factors (EEOC business necessity defense); (6) if not justified, eliminate the feature or practice causing the disparity; (7) document the analysis and remediation. Statistical significance should be tested: small sample sizes can produce apparent disparity from chance.
Disparate Impact — Approval Rates by Group
Group A
Group B
Group C
80% rule: Group C rate (41%) is 57% of Group A (72%) → disparate impact detected
Real-World Example
A major insurance company's AI pricing model charged significantly higher premiums to customers in certain zip codes. Analysis revealed that these zip codes correlated strongly with racial composition—predominantly minority neighborhoods paid premiums 22% higher than equivalent-risk customers in predominantly white neighborhoods. The zip code feature was not explicitly racial, but its correlation with race created actionable disparate impact. After the company was alerted by a state insurance regulator, they removed zip code from the pricing model and replaced it with hyper-local crime and accident statistics that maintained predictive accuracy without the demographic correlation. The company avoided formal enforcement action by self-remediating before a finding.
Common Mistakes
- ✕Assuming disparate impact only arises from explicit use of protected attributes—proxy variables commonly create disparate impact without any explicit discrimination
- ✕Testing for disparate impact only at the model level—the full pipeline (preprocessing, feature engineering, model, decision threshold) must be tested
- ✕Treating statistical significance as proof of no disparate impact—statistical tests may lack power with small samples; practical significance thresholds matter too
Related Terms
Algorithmic Fairness
Algorithmic fairness defines formal mathematical criteria for measuring and achieving equitable treatment across demographic groups in AI decision systems—including demographic parity, equalized odds, and individual fairness.
AI Bias
AI bias is the systematic tendency of AI models to produce unfair outcomes for certain groups—arising from skewed training data, biased features, or flawed objective functions—leading to discriminatory predictions or decisions.
Fairness Metrics
Fairness metrics are quantitative measures that evaluate how equitably an AI system treats different demographic groups—providing the mathematical foundation for detecting and reporting bias in model predictions.
Responsible AI
Responsible AI is a framework of organizational practices and principles—encompassing fairness, transparency, privacy, safety, and accountability—that guide how teams build and deploy AI systems that are trustworthy and beneficial.
EU AI Act
The EU AI Act is a comprehensive European Union regulation that classifies AI systems by risk level and imposes corresponding transparency, safety, and accountability requirements—the world's first major binding AI regulation with global compliance implications.
Ready to build your AI chatbot?
Put these concepts into practice with 99helpers — no code required.
Start free trial →