Regularization Techniques

Regularization reduces overfitting by discouraging overly complex models.

Common techniques

Technique	What it does	When useful
L2 (Ridge)	Shrinks weights smoothly	General stabilization
L1 (Lasso)	Pushes some weights to zero	Feature selection
Dropout	Randomly drops neurons during training	Deep networks
Early stopping	Stops when validation stops improving	Prevents late overfitting
Data augmentation	Adds transformed samples	Vision/audio tasks

Practical guidance

• Always monitor validation curves.
• Use cross-validation for robust estimates.
• Tune regularization strength gradually.