Outlier Detection: The IQR Rule, Z-Scores, and What to Do Next

One unusual value can distort a mean, inflate a standard deviation, and pull a regression line off course. Deciding whether a value is a genuine outlier — and what to do about it — is one of the most consequential judgment calls in any analysis, and it's a judgment call, not a mechanical formula, even though several formal methods exist to flag candidates.

Method 1: The 1.5×IQR Rule

Any value below the lower fence or above the upper fence is flagged as a potential outlier. This method, covered in detail in our percentiles and quartiles guide, is resistant to the outliers it's trying to detect — since Q1 and Q3 only depend on position in the sorted data, not the magnitude of extreme values, a single huge outlier doesn't distort the fences used to flag it. This makes it the more robust default choice, especially for skewed data.

Method 2: The Z-Score Rule

Values more than 3 standard deviations from the mean are flagged. Unlike the IQR method, this one assumes the data is roughly normally distributed, and — because it uses the mean and standard deviation directly — it's itself sensitive to the outliers it's trying to detect: a single extreme value inflates the standard deviation, which can mask other genuine outliers by making the flagging threshold wider than it should be. This is exactly why the IQR method is generally preferred for skewed or non-normal data.

Worked Example: Both Methods on the Same Data

Problem: Nine employees' commute times (minutes): 18, 22, 20, 25, 21, 19, 23, 24, 65. Flag outliers using both methods.

IQR method: sorted: 18,19,20,21,22,23,24,25,65. Median=22. Lower half: 18,19,20,21 → Q1=19.5. Upper half: 23,24,25,65 → Q3=24.5. IQR = 24.5−19.5 = 5. Fences: 19.5−7.5=12, 24.5+7.5=32. 65 is flagged (above 32).

Z-score method: mean ≈ 26.3, SD ≈ 14.6 (inflated by the 65). z for 65 = (65−26.3)/14.6 ≈ 2.65 — this does NOT clear the |z|>3 threshold, despite 65 being an obvious outlier by inspection and by the IQR method. This is the exact masking effect described above: the outlier itself inflated the standard deviation enough to make the z-score threshold too lenient to flag it.

Grubbs' test (a more rigorous version of the z-score idea): rather than a fixed cutoff of 3 regardless of sample size, Grubbs' test computes G = max|xᵢ − x̄| / s — exactly the same 2.65 from the z-score calculation above, computed to more decimal places as G ≈ 2.634 — and compares it against a critical value computed specifically for n=9, from the Grubbs' test critical values table: 2.215 at α=0.05. Since 2.634 exceeds 2.215, Grubbs' test does flag 65 as a statistically significant outlier, succeeding exactly where the arbitrary |z|>3 cutoff failed — because its critical value was built for this specific sample size rather than borrowed as a one-size-fits-all rule of thumb.

How Outliers Affect Different Statistics

Outliers don't affect every statistic equally, and knowing which ones are resistant helps decide how to handle a flagged value:

• Mean: highly sensitive — a single extreme value can shift it substantially.
• Median: highly resistant — only the position of values matters, not their magnitude.
• Standard deviation / variance: highly sensitive — squaring deviations amplifies the influence of extreme values.
• IQR: highly resistant — depends only on the middle 50% of data, ignoring the extremes entirely.
• Correlation and regression slope: can be substantially distorted by even a single influential outlier, as covered in Pearson correlation.

What to Actually Do With a Confirmed Outlier

Once a flagged value is investigated, there are several legitimate paths, each appropriate in different circumstances: correct it if it's a verified data-entry error; remove it if it's confirmed invalid (like a sensor malfunction reading) and clearly document that removal; keep it and report both the outlier-inclusive and outlier-excluded results side by side, if there's genuine uncertainty about validity; or use a statistical method specifically designed to be robust to outliers (like the median instead of the mean, or a trimmed mean) rather than removing data at all. What's never legitimate is removing a data point purely because it makes a desired result look better or a p-value cross a threshold — a practice sometimes called "cleaning until it confesses," and a serious violation of research integrity.

Real-World Uses

Fraud detection: unusual transaction amounts or patterns are flagged using outlier-detection logic as a first-pass screen, though a flagged transaction still requires human or further automated investigation before any action.

Sensor and equipment monitoring: a sudden outlier reading from industrial equipment can indicate either a genuine fault or a sensor malfunction — outlier detection is the first automated step, with root-cause investigation following.

Scientific data cleaning: checking for and investigating outliers is a standard, documented step in exploratory data analysis, before any formal statistical modelling begins.

Common Mistakes

Removing outliers to improve a result. Deleting inconvenient data points specifically because they make a desired conclusion less clear-cut, a p-value less significant, or an R² lower is a serious form of data manipulation, sometimes uncovered in cases of scientific misconduct — the decision to exclude a value must be justified independently of its effect on the result, not by that effect.

Using the z-score method on small or skewed samples without checking assumptions. As the worked example shows, the z-score method's own sensitivity to the outlier it's trying to detect can mask genuine extreme values, particularly in smaller samples where a single point has more influence over the mean and standard deviation.

Treating every flagged value identically. A value just past the fence and a value ten times past the fence are both "flagged," but they warrant different levels of scrutiny — some practitioners distinguish "mild" outliers (beyond 1.5×IQR) from "extreme" outliers (beyond 3×IQR) for exactly this reason, treating the latter with considerably more caution before inclusion.

Multivariate Outliers: A Value That's Only Unusual in Combination

All the methods above examine a single variable at a time. A value can be completely unremarkable on every individual variable while still being a genuine outlier in the RELATIONSHIP between variables — a person with a perfectly normal height and a perfectly normal weight for their age group could still be an outlier if that specific height-weight combination is unusual (very tall and very light, for instance). Detecting this kind of multivariate outlier requires methods that look at variables jointly, like examining a scatter plot's residuals in a regression context, rather than univariate methods like the IQR rule applied to each variable separately.

A Second Worked Example: Investigating Before Deciding

Problem: A retailer's daily sales figures average $12,000 with most days between $10,000-$14,000, but one day shows $95,000. The IQR method flags it clearly. What's the correct next step?

Before removing or keeping this value, investigate: was there a major promotional event, a bulk B2B order, or a data-entry error (perhaps a duplicate entry, or a misplaced decimal turning $9,500 into $95,000)? Each explanation calls for a different response — a genuine promotional spike is real, valid data that belongs in the analysis (though perhaps flagged separately as a special event rather than treated as a typical day); a data-entry error should be corrected; a duplicate entry should be removed. The IQR method's job was only to flag the candidate — the actual decision requires information the statistical method alone can't provide.