Where Statistics Actually Shows Up in Machine Learning

Machine learning is often taught and marketed as something separate from statistics — a newer, more computational discipline built on algorithms rather than formulas. In practice, the two are far more entangled than that framing suggests: nearly every core machine learning concept is a direct application, or a direct descendant, of ideas covered elsewhere on this site. This guide connects the dots explicitly, for anyone learning statistics who's wondered where it actually gets used in modern data science.

Training and Test Sets Are a Sampling Problem

Every machine learning workflow starts by splitting available data into a training set (used to fit the model) and a test set (held back to evaluate it). This split exists for exactly the reason covered in exploratory data analysis and the broader inference framework on this site: a model's performance on the data it was trained on tells you almost nothing about how it will perform on new data it hasn't seen, in the same way that a sample statistic only generalizes to a population when the sample was drawn representatively. A model that fits its training data extremely well but performs poorly on the held-out test set has overfit — memorized quirks specific to the training sample rather than learning the genuine underlying pattern, the machine-learning equivalent of a sample that happens not to represent its population well.

Bias-Variance Tradeoff: A Familiar Idea, New Name

Machine learning's "bias-variance tradeoff" is a direct application of the accuracy-versus-precision distinction that shows up throughout statistical estimation. Bias is systematic error — a model that's consistently wrong in the same direction, too simple to capture the real pattern (underfitting). Variance is how much a model's predictions would change if it were retrained on a different random sample of training data — a model with high variance is overly sensitive to the specific data it happened to see (overfitting). The tradeoff exists because reducing one tends to increase the other: a very simple model (like fitting a straight line, as in simple linear regression) has low variance but risks high bias if the real relationship is more complex; a very flexible model can drive bias down but often at the cost of high variance. Choosing model complexity is, underneath the machine-learning terminology, choosing a point on this same statistical tradeoff.

Worked Example: Evaluating a Classifier

Problem: A spam-detection model is tested on 1,000 emails, of which 60 are genuinely spam. The model flags 50 emails as spam. Of those 50 flags, 40 are correctly spam (true positives) and 10 are not actually spam (false positives). Of the 940 emails the model didn't flag, 20 were actually spam that it missed (false negatives), and the remaining 920 were correctly left unflagged (true negatives). Calculate precision, recall, accuracy, and F1 score.

This 2×2 layout — a confusion matrix — is the same cross-tabulation structure used in a chi-square test of independence, just applied to model predictions versus actual outcomes instead of two categorical variables.

Precision = 40 / (40+10) = 40/50 = 0.80 — of every email the model flagged as spam, 80% genuinely were spam.

Recall = 40 / (40+20) = 40/60 ≈ 0.667 — of every email that was genuinely spam, the model caught about 66.7% of them.

Accuracy = (TP+TN) / total = (40+930) / 1000 = 0.97 — the model was correct on 97% of all emails overall. Note how much higher this looks than precision or recall: accuracy is inflated here because the two classes are imbalanced (only 6% of emails are actually spam), a genuine statistical pitfall — a model that flagged nothing as spam at all would still score 94% accuracy while being completely useless, which is exactly why precision and recall matter alongside accuracy, not instead of checking it.

F1 = 2 × (0.80 × 0.667) / (0.80 + 0.667) = 2 × 0.5333 / 1.4667 ≈ 0.727. The F1 score is the harmonic mean of precision and recall — a single number that penalizes a model for doing badly on either measure, useful for comparing models when there's no single obvious way to weigh the two against each other.

A Model's Output Is Usually a Probability

Most classification models — including logistic regression, one of the simplest and most direct statistical models used in machine learning — don't actually output a hard yes/no decision internally. They output a probability (say, 0.83 probability of being spam), which is then compared against a threshold (commonly 0.5) to produce the final flagged/not-flagged decision. Moving that threshold trades precision against recall directly: raising it makes the model more conservative (higher precision, lower recall, since it only flags emails it's very confident about), and lowering it does the reverse. This threshold choice is a statistical decision, not a purely computational one, and should be set based on which type of error (false positive or false negative) is more costly in the specific application.

Where Distributions and Inference Show Up

Feature scaling and outlier handling lean directly on descriptive statistics — many algorithms perform better when input variables are standardized using the same z-score transformation covered in z-scores, and outliers are typically identified using the same IQR-based methods from outlier detection.

Confidence intervals around model performance apply the same logic as confidence intervals elsewhere on this site: a single accuracy number from one test set is a point estimate, and reporting a range (often via repeated resampling) gives a more honest picture of how much that number might vary if the test were run on a different sample.

A/B testing a model's real-world impact (does a new recommendation model actually increase click-through rate, once deployed?) is a direct application of the hypothesis-testing framework covered in A/B testing — comparing a treatment group (new model) against a control group (old model or no model) using the same statistical machinery.

Naive Bayes classifiers are, as the name suggests, a direct application of Bayes' theorem — computing the probability of each possible class given the observed features, using the same prior-times-likelihood structure as the medical testing example in that guide, just applied across many features at once with an independence assumption ("naive") that simplifies the calculation.

Common Mistakes

Evaluating a model only on training data. This measures how well a model memorized its training set, not how well it generalizes — always evaluate on a held-out test set the model never saw during training.

Trusting accuracy alone on imbalanced data. As the worked example above shows, accuracy can look excellent while a model is nearly useless when one class is rare — check precision, recall, and F1 (or similar metrics) whenever classes aren't roughly balanced.

Treating a model's probability output as a guaranteed truth. A model outputting "83% probability of spam" is making a calibrated estimate, not a certainty — the same distinction between an estimate and a guarantee that runs through every confidence interval and hypothesis test on this site.