The Friedman Test: Comparing 3+ Related Groups by Rank

The Kruskal-Wallis test compares three or more independent groups without assuming normality. The Friedman test handles the related-groups version of the same problem — when the same subjects (or matched blocks) are measured under three or more conditions, and a repeated-measures ANOVA's normality assumption is in doubt. It's the natural extension of the two-condition Wilcoxon signed-rank test to three or more conditions, in the same way Kruskal-Wallis extends Mann-Whitney.

How It Works

Within each subject (or block), the k conditions are ranked from 1 to k. If the conditions truly have no effect, each subject's ranking should be essentially a random shuffle of 1 through k, so — summed across n subjects — every condition's total rank should come out close to the same. The Friedman statistic Fr measures how far the conditions' rank totals actually spread apart:

where n is the number of subjects/blocks, k is the number of conditions, and Rⱼ is the sum, across all n subjects, of the ranks given to condition j.

Worked Example: Three Diets, Six Subjects

Problem: Six people each try three diets (A, B, C) for a month, with a month's break between diets. Weight loss (kg): every single one of the 6 subjects lost the most weight on Diet A, the second most on Diet B, and the least on Diet C. Within every subject, the within-subject ranks are the same: A=1 (best), B=2, C=3.

Rank totals across all 6 subjects: R_A = 6×1 = 6, R_B = 6×2 = 12, R_C = 6×3 = 18.

Fr = [12/(6×3×4)] × (6² + 12² + 18²) − 3×6×4 = (12/72) × (36+144+324) − 72 = 0.1667 × 504 − 72 = 84 − 72 = 12.

From the exact Friedman critical values table, for k=3 conditions and n=6 subjects, the critical values are 5.333 (α=0.10), 7.0 (α=0.05), and 9.0 (α=0.01). The observed Fr=12 clears all three — in fact, every single subject agreeing on the exact same ranking is such a strong pattern that the exact p-value is P(Fr≥12) = 6/46656 ≈ 0.00013. The diets differ significantly in their effect on weight loss.

Exact vs. Approximate Critical Values

Like Kruskal-Wallis, Fr is only approximately chi-square distributed with k−1 degrees of freedom, and that approximation needs enough subjects to be trustworthy. For k=3 conditions with 2 to 9 subjects, and k=4 conditions with 2 to 4 subjects, the exact null distribution can be found directly — every one of the (k!)ⁿ equally likely ways the n subjects could each independently rank the k conditions is enumerated (over 10 million for k=3, n=9), and the exact critical value read straight off that distribution. Beyond that range, the chi-square approximation with k−1 degrees of freedom is the standard, reliable fallback.

Real-World Uses

Repeated-measures experiments: the same subjects tested under three or more conditions (three drug doses, three website designs, three teaching methods) where a within-subject ranking is natural and between-subject differences would otherwise add noise.

Judge or rater agreement across multiple items: several judges each ranking the same set of three or more contestants, products, or proposals, where the interest is whether the items are ranked consistently rather than randomly.

Before/during/after designs: measuring the same subjects at three or more time points (before treatment, during, after) when the measurements are unlikely to be normally distributed or the sample is too small to check.

Common Mistakes

Using Friedman when the groups aren't actually related. If the "conditions" were measured on different, independent subjects rather than the same subjects (or matched blocks) under each condition, Kruskal-Wallis is the right test, not Friedman — mixing the two up ignores the very within-subject correlation Friedman is built to use.

Stopping at a significant result. As with ANOVA and Kruskal-Wallis, a significant Fr only says the conditions aren't all equivalent — a follow-up pairwise procedure (such as pairwise Wilcoxon signed-rank tests with a multiple-comparisons correction) is needed to identify which conditions actually differ.

Forgetting the tie correction. When a subject ranks two or more conditions equally, those ranks are averaged, same as any other rank-based test — and a correction factor is typically applied to Fr to account for the reduced variance ties introduce.