Overfitting, Underfitting, and Bias-Variance Tradeoff

Good models should generalize to unseen data, not just memorize training data.

Underfitting (high bias)

• Model too simple.
• High train error and high validation error.

Overfitting (high variance)

• Model too complex.
• Low train error but high validation error.

How to detect quickly

• Train loss down + validation loss up -> overfitting signal.
• Both losses high -> underfitting signal.

Goal

Find the balance where total error is minimized: Bias^2 + Variance + noise.