Overfitting

What is Overfitting?

The failure mode where a model fits its training data so closely — including the noise and quirks specific to that data — that it fails to generalise to new, unseen examples. It's usually a sign the model has grown needlessly complex for the underlying problem, which is exactly what regularisation and more training data both work to counteract.

The five clues

Engineerdle #64 described Overfitting with five clues, revealed one at a time from hardest to most direct. Each clue points at a different property of the concept.

1. 01A model's tendency to memorise instead of actually learn.
2. 02It performs beautifully on data it's already seen, and poorly on data it hasn't.
3. 03It often shows up as a model that's grown needlessly complex for the problem at hand.
4. 04Regularisation and more training data are two classic countermeasures.
5. 05A model fitting training data so closely, including its noise, that it fails to generalise to new data.

More Machine Learning concepts

• Neural NetworkPuzzle #10