Explaining predictions.
WHY IT MATTERS
Decisions affecting people require explanation Debugging requires understanding Trust requires it
WHAT GLOBAL INTERPRETATION EXPLAINS
How the model behaves overall.
WHAT LOCAL INTERPRETATION EXPLAINS
Why a particular prediction was made.
WHAT INTRINSICALLY INTERPRETABLE MODELS ARE
Linear models and shallow trees, where the mechanism is visible.
WHAT POST-HOC METHODS PROVIDE
Explanation of complex models after the fact.
WHAT THE COMMON APPROACHES ARE
Feature importance by permutation Contribution attribution per prediction Partial dependence, showing average effect of a feature Local approximation with a simple model
WHAT TO BE CAREFUL WITH
Explanations that are approximations, presented as fact Correlated features, where attribution is arbitrary Explanations that satisfy rather than inform
WHY THAT LAST POINT MATTERS
A plausible explanation for a wrong prediction is worse than none.
WHAT TO PREFER WHERE EXPLANATION IS REQUIRED
A simpler model that is genuinely interpretable.
WHAT TO ACCEPT
That accuracy and interpretability frequently trade off.
WHAT TO DOCUMENT
What the model uses, and what it does not.
