Learning the examples instead of the pattern.
WHAT OVERFITTING IS
A model that learned the training data too specifically, including its noise, and fails on anything new.
THE ANALOGY
A student who memorised past exam papers rather than understanding the subject.
Perfect on those papers, lost on a new question.
HOW IT SHOWS
Excellent performance on training data, poor performance on data the model has not seen.
WHY IT HAPPENS
Too little training data Too complex a model for the data available Training for too long
THE OPPOSITE PROBLEM
Underfitting: a model too simple to capture the pattern. Poor on everything.
HOW IT IS ADDRESSED
Holding back some data for testing, so performance on unseen examples is measurable Simplifying the model Gathering more data Stopping training at the right point
WHY YOU SHOULD KNOW THIS
A vendor reporting accuracy figures should be reporting them on data the model never saw.
If they cannot say, that is a question worth asking.
