Chi-Square Test: Goodness-of-Fit and Independence, With Worked Examples

Most hypothesis tests compare means or proportions — the chi-square test is built for a different question entirely: are these categorical frequencies what you'd expect, or is something systematically different going on? Does the distribution of eye colours in a genetics experiment match the predicted Mendelian ratios? Is voting preference actually independent of age group, or is there a real relationship? The chi-square test handles both questions with the same underlying logic.

The Core Formula

This is the same overall hypothesis testing framework used throughout this site — a null hypothesis, a significance level, a test statistic, and a decision rule — applied specifically to categorical, count-based data rather than the continuous means a t-test handles.

For every category (or cell in a table), subtract the expected frequency (E) from the observed frequency (O), square it, divide by E, and sum across every category. A χ² of 0 means observed exactly matches expected; larger values indicate a growing mismatch, which the test then evaluates against a chi-square distribution to determine how unlikely that mismatch would be if H₀ (no real difference from expectation) were true.

Type 1: Goodness-of-Fit Test

Tests whether observed frequencies across a single categorical variable match a specific hypothesized distribution.

Worked example: A die is rolled 120 times. Standard theory predicts each face appears 20 times (120/6) if the die is fair. Observed: 1→14, 2→18, 3→22, 4→25, 5→17, 6→24. Is the die fair, at α = 0.05?

H₀: the die is fair (each face equally likely). H₁: the die is not fair.

χ² = (14−20)²/20 + (18−20)²/20 + (22−20)²/20 + (25−20)²/20 + (17−20)²/20 + (24−20)²/20

= 36/20 + 4/20 + 4/20 + 25/20 + 9/20 + 16/20 = 1.8+0.2+0.2+1.25+0.45+0.8 = 4.7

Degrees of freedom = (number of categories − 1) = 6 − 1 = 5. Critical value at α=0.05, df=5 ≈ 11.07.

Decision: χ² (4.7) < critical value (11.07) → fail to reject H₀. There's no significant evidence the die is unfair.

Type 2: Test of Independence

Tests whether two categorical variables are independent of each other, using a contingency table (rows = one variable's categories, columns = the other's).

Worked example: Is preference for Product A vs. Product B independent of age group? Survey results:

Under 30: 60 prefer A, 40 prefer B (100 total). 30-and-over: 30 prefer A, 70 prefer B (100 total). Total: 90 prefer A, 110 prefer B, 200 total.

Expected count formula: E = (row total × column total) / grand total

E(Under 30, A) = (100×90)/200 = 45. E(Under 30, B) = (100×110)/200 = 55.

E(30+, A) = (100×90)/200 = 45. E(30+, B) = (100×110)/200 = 55.

χ² = (60−45)²/45 + (40−55)²/55 + (30−45)²/45 + (70−55)²/55

= 225/45 + 225/55 + 225/45 + 225/55 = 5 + 4.09 + 5 + 4.09 = 18.18

Degrees of freedom = (rows−1)(columns−1) = (2−1)(2−1) = 1. Critical value at α=0.05, df=1 ≈ 3.84.

Decision: χ² (18.18) > critical value (3.84) → reject H₀. There's strong evidence that product preference is NOT independent of age group — a real relationship exists.

Why Squaring, and Why Divide by Expected?

Squaring each (O−E) deviation, exactly as in variance, makes every term positive so they don't cancel out, and penalizes larger mismatches more heavily. Dividing by the expected value E standardizes the deviation relative to how large that category was expected to be — a raw difference of 10 matters far more for a category expected to contain 15 observations than for one expected to contain 1,000, and dividing by E accounts for this directly.

Effect Size for Chi-Square: Cramér's V

Like a t-test's p-value, a significant chi-square result tells you a relationship likely exists but not how strong it is — especially important since χ² grows with sample size even for a fixed underlying effect. Cramér's V, ranging from 0 (no association) to 1 (perfect association), standardizes the effect size so it can be compared across different sample sizes and table dimensions, filling the same role Cohen's d fills for a t-test.

Common Real-World Uses

Genetics: testing observed offspring trait ratios against Mendelian predictions is a textbook goodness-of-fit application, and was in fact one of the test's earliest applications historically.

Market research: testing whether customer satisfaction category is independent of store location, or purchase category is independent of customer demographic, uses the independence form directly.

Quality control: testing whether defect types are independent of production line or shift uses the same contingency-table logic to flag whether a specific line or shift has a genuinely different defect profile, not just random variation.

A/B testing with categorical outcomes: comparing conversion across more than two variants, or comparing several discrete outcome categories (not just converted/didn't) between test groups, often uses a chi-square test rather than a simple two-proportion test.

Common Mistakes

Ignoring the expected-count rule. Running a standard chi-square test when several expected cell counts fall below 5 produces an unreliable p-value — the chi-square distribution is itself an approximation that only holds well under this condition. Combining sparse categories together, or switching to Fisher's exact test, are the standard fixes.

Confusing goodness-of-fit with independence. These use the identical formula but test fundamentally different questions and require different expected-count calculations — goodness-of-fit compares one variable's distribution to a theoretical expectation; independence compares two variables against each other within the same sample. Mixing up which one applies to a given research question is a common conceptual error, even when the arithmetic itself is executed correctly.

Treating chi-square as proof of causation. A significant test of independence shows an association between two categorical variables — exactly like correlation, it says nothing on its own about which variable (if either) causes the other, or whether a third confounding variable explains both.

Applying chi-square to non-independent observations. Like most classical hypothesis tests, chi-square assumes each observation is independent of the others. Counting the same individual in multiple cells (for instance, surveying the same person more than once and treating each response as a separate observation) violates this assumption and inflates the apparent significance of the result.

A Second Worked Example: Small Expected Counts

Problem: A rare-disease study compares treatment outcome (recovered/not) between a new drug (n=8) and placebo (n=8). Only 2 patients recovered on placebo and 6 recovered on the new drug. Is a standard chi-square test appropriate here?

Expected count for "placebo, recovered" = (row total × column total)/grand total = (8 × 8)/16 = 4 — already below the rule-of-thumb minimum of 5 for at least one cell, and with such small totals, several cells will likely fall short. A standard chi-square test's p-value would not be reliable here; Fisher's exact test, which calculates an exact probability rather than relying on the chi-square distribution's large-sample approximation, is the appropriate choice for this small a sample.