Machine learning (ML) is how most modern AI systems learn: instead of writing every rule by hand, you show the model examples and it picks up the pattern. This shift exists because many real-world problems are too messy to describe with manual rules alone.
You already do a form of machine learning every day. After enough rainy mornings, you bring an umbrella without deriving meteorology from first principles. Your brain updates from experience. In software, experience is data: features (inputs) and, often, labels (desired outputs).
Three questions organize almost every ML project:
If you cannot answer (3), you are not doing ML engineering — you are fitting a curve and hoping.
Learning from data. A model is a function with adjustable parameters. Training searches for parameter values that make predictions match reality — usually by minimizing a loss (squared error, cross-entropy, etc.). After training, you run inference on new inputs. The same skeleton covers linear regression and large neural nets; scale and architecture change, the loop does not.
Three broad paradigms:
| Plain-English idea | When to use it |
|---|---|
| Supervised learning — inputs plus labels; predict labels for new inputs | Spam filters, credit scoring, image triage |
| Unsupervised learning — inputs only; find hidden structure | Customer segments, anomaly hints, compression |
| Reinforcement learning (RL) — actions plus rewards over time; learn a policy | Game-playing agents, some robotics and recommendation loops |
Semi-supervised and self-supervised methods blur the lines: they use lots of unlabeled data plus a pretext task (predict the next token, fill in a masked image) to learn useful representations.
Regression vs classification (supervised).
| Plain-English idea | When to use it |
|---|---|
| Regression — predict a continuous number (price, temperature, latency) | Forecasting quantities |
| Binary classification — predict one of two labels (spam/ham, disease/no disease) | Yes/no decisions |
| Multi-class classification — predict one of many labels (which digit, which topic) | Picking among several categories |
Same idea — map input to output — different output type and usually different loss.
Train / test split. If you grade a student only on questions they memorized, you overestimate skill. Likewise, if you evaluate a model on the same rows used to fit it, you overestimate generalization. Hold out a test set (and often a validation set for tuning) so metrics reflect performance on unseen data. Touch the test set once for final reporting; use validation for model selection.
Fit a line with the closed-form least-squares solution (stdlib only), then compare to a naive average baseline. No scikit-learn.
# Predict y from x: y ~= a + b*x (ordinary least squares, 1-D)
xs = [1.0, 2.0, 3.0, 4.0, 5.0]
ys = [2.1, 3.9, 6.2, 7.8, 10.1]
n = len(xs)
mean_x = sum(xs) / n
mean_y = sum(ys) / n
# Closed form: b = Cov(x,y) / Var(x); a = mean_y - b * mean_x
var_x = sum((x - mean_x) ** 2 for x in xs)
cov_xy = sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys))
b = cov_xy / var_x
a = mean_y - b * mean_x
def predict(x: float) -> float:
return a + b * x
# Average baseline: always predict mean_y (ignores x)
baseline = mean_y
# Hold out last point as a tiny "test" example
x_test, y_test = xs[-1], ys[-1]
err_model = (predict(x_test) - y_test) ** 2
err_base = (baseline - y_test) ** 2
print(f"fit: y = {a:.3f} + {b:.3f}*x")
print(f"test MSE model={err_model:.3f} baseline={err_base:.3f}")Even this toy fit shows the ML loop: choose a hypothesis class (a line), fit on train data, compare to a baseline, evaluate on held-out points. Real projects add more features, regularization, cross-validation, and careful splits — the skeleton stays the same. Always ship a baseline first; if a fancy model cannot beat "predict the mean," stop and fix the data or the problem statement.
Machine learning fits models from data so systems generalize to new inputs — via supervised, unsupervised, or reinforcement paradigms, evaluated on held-out data.