Tukey's HSD: Finding Which Groups Actually Differ After ANOVA

One-way ANOVA answers a narrow question: is at least one group mean different from the others? A significant result doesn't say which group, or how many. Tukey's Honestly Significant Difference (HSD) test is the standard follow-up — a post-hoc test that makes every pairwise comparison between groups while keeping the overall false-positive rate under control, avoiding exactly the multiple comparisons problem that running separate, uncorrected t-tests for every pair would create. It answers the practical question a significant ANOVA result raises but can't resolve on its own: given that some group is genuinely different, which one — or ones — actually is?

Why Not Just Run Pairwise T-Tests?

With k=3 groups, there are 3 possible pairwise comparisons; with k=6 groups, 15. Running an ordinary t-test for each pair, each at α=0.05, reintroduces the exact multiple-comparisons inflation that ANOVA was built to avoid in the first place — with enough pairs, the chance of at least one false "significant" difference climbs well above the stated 5%. Tukey's HSD solves this by using a single, specially calibrated critical value for every pairwise comparison in the set, keeping the overall false-positive rate across all comparisons at the target level, rather than letting each individual comparison carry its own separate 5% risk.

The Formula

Here q is the studentized range statistic — a critical value from its own reference distribution, looked up (or computed by software) for a given significance level α, number of groups k, and the within-group degrees of freedom from the original ANOVA. MS_within is the ANOVA's within-group mean square (a pooled estimate of the common variance shared across groups), and n is the number of observations per group (assuming equal group sizes, the simplest case). Any pair of group means whose absolute difference exceeds HSD is declared significantly different.

Worked Example

Problem: Using the same fertilizer data from one-way ANOVA — Fertilizer A: 12,14,13,15 (mean 13.5); Fertilizer B: 16,18,17,19 (mean 17.5); Fertilizer C: 10,11,9,12 (mean 10.5); n=4 plants per group, and MS_within = SSW/df_within = 15/9 ≈ 1.667 — determine which pairs of fertilizers produced significantly different growth.

With k=3 groups and df_within=9, the studentized range critical value at α=0.05 is q ≈ 3.95 (you can look up any combination of k and df in our studentized range (Tukey q) table).

HSD = 3.95 × √(1.667/4) = 3.95 × √0.4167 = 3.95 × 0.6455 ≈ 2.55

Any pair of group means differing by more than 2.55 is significantly different. Comparing all three pairs:

|A − B| = |13.5 − 17.5| = 4.0 > 2.55 → significant

|A − C| = |13.5 − 10.5| = 3.0 > 2.55 → significant

|B − C| = |17.5 − 10.5| = 7.0 > 2.55 → significant

All three pairwise differences exceed the HSD threshold — every fertilizer produces significantly different growth from every other one, not just "at least one difference exists somewhere" as the original ANOVA result alone established. This full picture (Fertilizer B best, A second, C worst, and every gap statistically real) is exactly the additional information a significant ANOVA result on its own can't provide.

Reading the Result Correctly

It's worth being precise about the claim Tukey's HSD actually supports: it identifies which specific pairs of means differ significantly, while maintaining the overall false-positive guarantee across the whole family of comparisons made. It does not, on its own, rank the size or practical importance of those differences beyond the raw mean gap — a statistically significant 4.0 cm difference and a statistically significant 7.0 cm difference are both "significant," but the 7.0 cm gap represents a substantially larger practical effect, worth noting alongside the significance result itself rather than treating "significant" as a single undifferentiated verdict. Reporting the actual mean differences and their confidence intervals, not just a list of which pairs cleared the threshold, gives a reader a much fuller picture than a bare significant/not-significant label for each pair.

When Group Sizes Are Unequal

The formula shown above assumes equal sample sizes across groups (the simplest, "textbook" case). When group sizes differ, a modified version — the Tukey-Kramer method — adjusts the standard error calculation separately for each specific pair being compared, using each pair's own two sample sizes rather than a single shared n. The core logic (compare each pairwise difference against a calibrated critical threshold) stays identical; only the standard error formula changes to accommodate the unequal group sizes.

Real-World Uses

Agricultural and clinical trials: exactly the fertilizer scenario worked through above — once ANOVA establishes that at least one treatment differs from the others, Tukey's HSD is the standard way to identify the specific superior (or inferior) treatments researchers and practitioners actually need to know about before making a recommendation.

Marketing and product testing: comparing customer response across several different product variants, ad creatives, or pricing tiers commonly uses ANOVA followed by Tukey's HSD to pinpoint exactly which variants perform differently from which others, rather than stopping at the less actionable "something differs somewhere" conclusion.

Manufacturing process comparison: testing output quality across several machines, shifts, or suppliers follows the same two-step pattern — ANOVA flags whether a real difference exists anywhere in the set, and Tukey's HSD narrows that down to the specific pairs worth investigating or acting on.

Common Mistakes

Running Tukey's HSD without a significant ANOVA result first. Standard practice is to treat the post-hoc test as a follow-up specifically triggered by a significant overall ANOVA result — running pairwise comparisons regardless of the initial ANOVA outcome undermines the very error-rate control the two-step procedure is designed to provide.

Using an ordinary t-distribution critical value instead of the studentized range statistic q. These are genuinely different reference distributions — q is specifically calibrated for the "compare the largest gap among several group means" scenario, and using an ordinary t critical value in its place reintroduces the same false-positive inflation Tukey's method exists to prevent.