Measuring numerical and ordered predictions.
WHAT MEAN ABSOLUTE ERROR MEASURES
Average size of error, in the original units.
WHAT SQUARED ERROR MEASURES
Average squared error, penalising large errors more.
WHEN TO PREFER SQUARED ERROR
When large errors are disproportionately costly.
WHEN TO PREFER ABSOLUTE ERROR
When they are not, and when outliers should not dominate.
WHAT PERCENTAGE ERROR PROVIDES
Scale-independent comparison.
WHERE IT BREAKS
Near zero values, where it becomes enormous or undefined.
WHAT THE COEFFICIENT OF DETERMINATION MEASURES
Proportion of variance explained.
WHY IT MISLEADS
It can be high while predictions are useless in practical terms.
WHAT TO ALWAYS REPORT
Error in the units people understand.
WHAT RANKING METRICS MEASURE
Whether relevant items appear near the top.
WHAT THE COMMON ONES CONSIDER
Precision within the top few Position of the first relevant item Gain discounted by position
WHY POSITION MATTERS
Users see the top few and nothing else.
WHAT TO CHOOSE
The metric reflecting how the output is actually consumed.
WHAT TO NEVER DO
Report a metric nobody can interpret in business terms.
