Deep learning is machine learning with multi-layer neural networks that learn their own features from raw inputs. It exists because many problems — images, speech, language — are too rich to hand-engineer every signal.
Classical ML often looks like: engineer features -> train a shallow model (logistic regression, random forest). Deep learning folds feature engineering into the model. Early layers detect simple patterns; deeper layers recombine them — from pixels to textures to objects, or from tokens to phrases to meaning.
"Depth" means many successive nonlinear transforms. Each layer is usually a linear map (weights plus bias) followed by a nonlinearity (ReLU, GELU, etc.). With nonlinearities, composition can carve intricate decision regions and rich embeddings.
Why care as a practitioner? Because many GenAI components are deep nets: transformers, convolutional backbones, diffusion U-Nets. Understanding depth and representation learning clarifies why these models need large data, why fine-tuning works, and why a shallow model on raw pixels usually fails.
Neural networks. A network is a composition of layers. For a vector x:
h1 = activation(W1 * x + b1)
h2 = activation(W2 * h1 + b2)
...
y_hat = WL * h(L-1) + bLTraining adjusts all weights and biases with gradient-based optimization (backpropagation plus stochastic gradient descent (SGD) or Adam) to reduce a loss on labeled (or self-supervised) data. Modern stacks add normalization, residual skip connections, attention, and careful initialization so gradients can travel through dozens or hundreds of layers.
Why depth helps. A deep stack can approximate complex functions more efficiently than a single wide shallow layer for many structured problems: hierarchical composition matches how language and vision are organized. Empirically, with enough data and compute, deeper models keep improving where hand-built features plateau. Depth is not free — optimization gets harder, and overfitting risk rises — but for raw signals it is often the winning approach.
Relationship to neural networks. Deep learning is not a different species from "neural nets" — it is the practice of training deep ones (many layers), often with modern tricks: better activations, normalization, residual connections, attention, large-scale data, and GPUs/TPUs. A single perceptron is a neural network; a 96-layer transformer is deep learning. Same family, different scale and architecture.
When deep learning wins vs classical ML:
| Plain-English idea | When to use it |
|---|---|
| Classical ML — gradient boosting, random forests on well-engineered tabular features | Small or medium tabular data; you need strong interpretability and fast iteration |
| Deep learning — multi-layer nets on raw or lightly processed inputs | Raw high-dimensional inputs (images, audio, text); end-to-end accuracy matters and you have data and compute |
Many strong systems are hybrids: classical models on engineered features plus deep embeddings from a neural net (for example, gradient boosting on tabular columns concatenated with a text embedding).
One forward pass of a tiny 2-layer network with NumPy — no training, just the computation graph you would later differentiate.
import numpy as np
rng = np.random.default_rng(0)
# Input: batch of 4 examples, each with 3 features
x = rng.normal(size=(4, 3))
# Layer 1: 3 -> 5, then ReLU
W1 = rng.normal(scale=0.5, size=(3, 5))
b1 = np.zeros(5)
h = np.maximum(0, x @ W1 + b1) # ReLU
# Layer 2: 5 -> 2 logits (e.g. 2-class scores)
W2 = rng.normal(scale=0.5, size=(5, 2))
b2 = np.zeros(2)
logits = h @ W2 + b2
# Softmax for probabilities (numerically stable)
logits = logits - logits.max(axis=1, keepdims=True)
probs = np.exp(logits)
probs = probs / probs.sum(axis=1, keepdims=True)
print("hidden shape:", h.shape)
print("probs:\n", np.round(probs, 3))Training would add a loss (e.g. cross-entropy vs labels) and update W1, b1, W2, b2 with gradients. The forward pass above is the core "what deep learning computes" before any optimizer enters the picture. Frameworks like PyTorch automate the backward pass; conceptually you are still chaining matrix multiplies and nonlinearities.
Deep learning stacks nonlinear neural layers to learn hierarchical representations, excelling on raw high-dimensional data when classical feature engineering hits a wall.