Models that treat people equitably.
WHY IT MATTERS
Models make or influence decisions about people, and errors distribute unevenly.
WHERE UNFAIRNESS ORIGINATES
Historical data reflecting past discrimination Groups underrepresented in training data Features correlating with protected characteristics Labels reflecting biased human decisions Deployment contexts unlike training conditions
WHY REMOVING PROTECTED ATTRIBUTES IS INSUFFICIENT
Other features correlate with them, and the model learns the same pattern.
WHAT FAIRNESS DEFINITIONS EXIST
Equal outcome rates across groups Equal error rates across groups Equal treatment of similar individuals Calibration within each group
WHAT THE UNCOMFORTABLE FACT IS
Several of these cannot hold simultaneously except in special cases.
WHAT THAT MEANS
Fairness requires choosing which definition applies, deliberately.
WHAT TO MEASURE
Performance disaggregated by group, always.
WHAT THAT REQUIRES
Knowing group membership, which may itself be sensitive.
WHAT TO DOCUMENT
The definition chosen, and why.
WHAT TO NEVER DO
Claim a model is fair without stating what that means.
WHAT OBLIGATIONS MAY APPLY
Anti-discrimination law, which is specific. Take advice on your position.
