Leverage, Outliers, and Cook's Distance: Finding Influential Points in Regression

A regression line is supposed to summarize a whole dataset, but some points have far more say in where the line goes than others. A single observation can tilt the slope, change the intercept, and even flip the apparent story, while sitting quietly in the data table looking unremarkable. Detecting such points takes three related ideas that are easy to confuse: outliers, leverage, and influence. Cook's distance is the standard statistic that combines them into a single measure of how much any one point moves the fitted model.

Three Different Ideas

An outlier in regression is a point with an unusually large residual: its y-value is far from what the fitted line predicts. Pinning down that idea for one variable is covered in outlier detection; in regression, the question is always relative to the line.

A high-leverage point has an unusual x-value, far from the other x-values. It has the potential to pull the line toward itself, whether or not it actually does. In simple regression, the leverage of point i is:

Leverages always sum to the number of model parameters, so the average leverage is p ÷ n. A common warning threshold is 2p ÷ n.

An influential point is one whose removal would noticeably change the fitted model. Influence requires both ingredients at once: a point far out in x has the power to move the line (leverage), and a point that also disagrees with the trend of the rest (a sizable residual) uses that power. A high-leverage point that falls right on the trend of the other points is harmless and even helpful, since it anchors the slope with extra precision.

Worked Example

Problem: Six observations are recorded. Five sit close to a rising trend; the sixth has a much larger x-value and a low y-value.

Point	1	2	3	4	5	6
x	1	2	3	4	5	12
y	3	5	6	9	10	8

Step 1 — Fit the line to all six points. Here x̄ = 27 ÷ 6 = 4.5 and ȳ = 41 ÷ 6 ≈ 6.833. Then Sxx = 199 − 6(4.5)² = 77.5 and Sxy = 213 − 6(4.5)(6.833) = 28.5, exactly as in simple linear regression. The slope is 28.5 ÷ 77.5 ≈ 0.368 and the intercept is 6.833 − 0.368(4.5) ≈ 5.18, giving ŷ = 5.18 + 0.368x, with R² ≈ 0.30.

Step 2 — Compute each point's leverage. For point 6, h₆ = 1/6 + (12 − 4.5)² ÷ 77.5 = 0.167 + 56.25 ÷ 77.5 ≈ 0.892. For point 1, h₁ = 1/6 + (1 − 4.5)² ÷ 77.5 ≈ 0.325. The warning threshold is 2p ÷ n = 2(2) ÷ 6 ≈ 0.667, and only point 6 exceeds it.

Step 3 — Compute Cook's distance. The formula combines the residual and the leverage:

The residual sum of squares is about 24.35 on n − p = 4 degrees of freedom, so MSE ≈ 6.088. For point 6, the fitted value is 5.18 + 0.368(12) ≈ 9.59, so e₆ = 8 − 9.59 ≈ −1.59. Then D₆ = [(1.59)² ÷ (2 × 6.088)] × [0.892 ÷ (0.108)²] ≈ 0.208 × 77.2 ≈ 16.05. The full set of diagnostics is:

Point	x	y	Residual	Leverage	Cook's D
1	1	3	−2.55	0.325	0.379
2	2	5	−0.91	0.247	0.030
3	3	6	−0.28	0.196	0.002
4	4	9	2.35	0.170	0.112
5	5	10	2.98	0.170	0.180
6	12	8	−1.59	0.892	16.05

Point 6 stands far apart from every other point. Common rules of thumb flag a Cook's distance above 1, or above 4 ÷ n (here about 0.67); point 6 clears both by a wide margin, while no other point comes close.

The Proof: Refit Without It

Cook's distance is a prediction of how much a point matters; refitting confirms it. Using only points 1 to 5, the line becomes ŷ = 1.2 + 1.8x with R² ≈ 0.98. Adding the sixth point drops the slope from 1.80 to 0.37, roughly a fifth of its value, and R² collapses from 0.98 to 0.30. One observation out of six has flipped the conclusion from "y rises steeply with x" to "y barely responds to x."

Why the Residual Alone Misleads

Look again at point 6's residual in the table: −1.59, smaller than the residuals of points 1, 4, and 5. By residuals alone, it would not stand out at all. The reason is that a high-leverage point drags the line toward itself, so the line ends up close to it and its residual shrinks. Had the line stayed on the trend set by the first five points, point 6 would have missed by 8 − (1.2 + 1.8 × 12) = 8 − 22.8 = −14.8 units. This is why checking residual plots alone, as in the residual analysis section of the simple regression article, can miss the most damaging points, and why leverage and Cook's distance are needed alongside them.

What to Do With an Influential Point

Finding one is the start of the work, not a licence to delete it. The sensible steps run roughly as follows:

• Check for an error. A typo, unit mix-up, or instrument glitch is the most common explanation, and a documented correction is legitimate.
• Ask whether it belongs to the same population. A point from a different regime, such as a much larger company in a study of small businesses, may not be governed by the same relationship.
• Report the model both ways. Present results with and without the point, so readers see how much rests on it.
• Consider more robust methods. Rank-based measures like Spearman's correlation are less swayed by extreme values, and robust regression methods downweight them.

Real-World Uses

Economics and finance: a single crisis year in a decades-long dataset can dominate a fitted trend, and influence diagnostics reveal when a headline coefficient depends on that one period.

Medical and biological studies: with small samples, one unusual patient or animal can drive a reported dose-response slope, so influence checks are a routine part of model validation.

Model auditing in data science: before deploying a model built with multiple regression, analysts scan Cook's distances to make sure the fit is not held up by a handful of rows.

Common Mistakes

Deleting influential points automatically. Removing points simply because they change the result is data dredging. A point should be excluded only with a substantive reason, disclosed openly.

Treating leverage and outlyingness as the same thing. A high-leverage point on the trend is harmless, and an outlier in the middle of the x-range moves the line only slightly. Influence needs both.

Relying on a fixed cutoff mechanically. Rules like D above 1 or above 4 ÷ n are guides, not laws. In very small datasets nearly every point looks influential, so plotting the data and refitting matters more than any threshold.

Checking only for a single point. Two influential points close together can mask each other, since removing either one alone changes little.

What is a good Cook's distance cutoff?

Two guidelines are common: flag any value above 1, or any value above 4 divided by the sample size. The second is more sensitive and is usually a prompt to look at the point, not a verdict.

Can a point be influential without being an outlier?

Yes, as the worked example shows. Point 6 had an unremarkable residual but enormous influence, because its extreme x-value gave it high leverage. Influence does not require a large residual.

Does the same idea work with several predictors?

Yes. Leverage generalizes to distance from the center of the whole set of predictors, and the Cook's distance formula is unchanged apart from p, which becomes the number of model coefficients. It applies to any multiple regression model.

How does this relate to Anscombe's quartet?

Anscombe's quartet, discussed in simple linear regression, shows four datasets with nearly identical fitted lines but very different structures, including one where a single extreme point creates the entire apparent relationship. Influence diagnostics are the numerical counterpart to the visual lesson.