Bias Testing, Translated

"Bias audit" sounds technical. The question it asks is not. Strip away the jargon and a bias audit asks: does this tool treat different groups of people differently, in ways that matter?

Here's what that looks like in practice. You run the tool against test cases across groups — different ages, genders, races, zip codes — and compare outcomes. Does the resume screener advance men and women at similar rates? Does the lending model approve comparable applicants comparably? Small differences are expected. Big, consistent gaps are a finding.

Two myths to retire. One: "our data is neutral." Data reflects the world that produced it, biases included. Two: "we don't use protected characteristics, so we're fine." Models find proxies — zip codes, names, shopping patterns. Testing is how you catch what the inputs hide.

I explain frameworks; I don't give legal counsel.