The most regulated application.
WHAT IS AT STAKE
Access to credit affects people materially. Errors and bias have serious consequences.
WHAT REGULATION GENERALLY REQUIRES
An explanation for adverse decisions Testing for discriminatory outcomes Human review, particularly on appeal Documentation of the model and its basis
THE PROXY PROBLEM
Removing protected characteristics does not remove bias.
Postcode, employer, school and spending patterns all correlate with characteristics you are not permitted to use.
A model with no race field can still produce racially disparate outcomes.
WHAT TO TEST
Outcomes by group, not just overall accuracy.
If approval rates differ substantially between groups with similar risk profiles, that requires investigation.
THE HISTORICAL DATA PROBLEM
A model trained on past lending decisions learns past preferences, including discriminatory ones.
WHAT MUST REMAIN HUMAN
Appeals Exceptions Anything where the model is uncertain
WHAT TO DOCUMENT
The basis of the model, the testing performed, and the review process.
