Disparate Impact, in Plain English

"Disparate impact" sounds like legal jargon, but the idea is simple: a rule can treat everyone the same on paper and still hurt one group far more than others.

The classic pattern: a requirement that looks neutral — say, a minimum height for a job, or a specific diploma — screens out far more women, or more people from certain backgrounds, without being necessary for the work. Nobody wrote the rule to discriminate. The impact discriminates anyway.

In plain English: it's not about what the rule says. It's about what the rule does.

AI systems are disparate-impact machines waiting to happen. A model trained on historical data will happily learn the neutral-looking patterns that produced unequal outcomes before — zip codes, employment gaps, school names — and apply them at scale, all while the rulebook looks perfectly fair.

This is why outcome testing matters more than intention-checking. Don't ask "did we mean to be fair?" Ask "who does this actually affect, and is the difference justifiable?" If you can't explain the gap, you can't defend the system.

I explain frameworks; I don't give legal counsel.