The Likelihood Ratio Test: Comparing Two Models Directly

Building on maximum likelihood estimation: once a model's parameters have been fit by finding the maximum likelihood, a natural next question follows — how much better does a more flexible model actually explain the data compared to a simpler, restricted one? The likelihood ratio test answers exactly this, comparing two nested models' maximized likelihoods directly, using the same likelihood framework already introduced there.

Nested Models

The likelihood ratio test compares two models where one is a special, restricted case of the other — the simpler model can be obtained from the more flexible one by fixing one or more parameters to specific values. For example, "the coin is fair" (p fixed at exactly 0.5) is a restricted special case of "the coin has some unknown bias p" (p free to take any value) — the fair-coin model is nested inside the general model.

The Test Statistic

Here log L(restricted) is the maximized log-likelihood under the simpler, restricted model, and log L(full) is the maximized log-likelihood under the more flexible model. Since the full model can always fit at least as well as its own restricted special case, log L(full) is always at least as large as log L(restricted), making LR always non-negative. Under the null hypothesis that the restricted model is correct, LR follows a chi-square distribution, with degrees of freedom equal to the difference in the number of free parameters between the two models.

Worked Example: Is the Coin Actually Fair?

Problem: Using the same data as the maximum likelihood estimation coin-flip example — 7 heads in 10 flips — test H₀: p=0.5 (the coin is fair) against H₁: p≠0.5 (the coin has some other, unknown bias), using the likelihood ratio test.

The restricted model fixes p=0.5, giving log L(restricted) ≈ −2.144 (computed directly from the binomial log-likelihood formula at p=0.5). The full model uses the MLE estimate p̂=0.7 found previously, giving log L(full) ≈ −1.321 — necessarily higher, since 0.7 was specifically chosen to maximize the likelihood.

LR = −2 × [−2.144 − (−1.321)] = −2 × (−0.823) ≈ 1.646

The two models differ by 1 free parameter (the restricted model fixes p; the full model estimates it), so LR is compared against a chi-square distribution with 1 degree of freedom. This gives a p-value of approximately 0.1996 — well above the conventional 0.05 threshold. With only 10 flips, there isn't strong enough evidence to reject the fair-coin hypothesis, even though the observed proportion (0.7) looks noticeably different from 0.5 on its face; the sample is simply too small to distinguish "genuinely biased" from "fair coin showing ordinary sampling variation" with confidence.

Why the Test Statistic Follows a Chi-Square Distribution

This result — that −2 times the log-likelihood difference between nested models follows a chi-square distribution under the null hypothesis — is a general large-sample mathematical result (known as Wilks' theorem), not something specific to the coin-flip example. It's exactly what makes the likelihood ratio test broadly applicable: the same LR-statistic-to-chi-square-p-value pipeline works for comparing any two nested models, however complex, as long as both were fit via maximum likelihood — logistic regression models with different sets of predictors, Poisson regression models, and many others beyond the simple example shown here.

Connection to Other Tests on This Site

The likelihood ratio test is a close relative of the chi-square test already covered elsewhere on this site — both ultimately compare an observed pattern against an expected one using a chi-square reference distribution, though the chi-square goodness-of-fit test works directly with observed-versus-expected counts, while the likelihood ratio test works one level more abstractly, with the likelihoods of two competing models. For a simple case like testing a single proportion against a hypothesized value, the likelihood ratio test and other familiar approaches (like the exact binomial test, or a normal-approximation z-test on the proportion) tend to agree closely, especially as sample size grows — the likelihood ratio framework's real advantage shows up in more complex models where no simpler alternative test is available.

Real-World Uses

Comparing nested regression models: testing whether adding a new predictor (or group of predictors) to a logistic or Poisson regression model genuinely improves the fit, beyond what's expected from the added flexibility alone, is a standard application of the likelihood ratio test in applied regression work.

Genetics and molecular biology: comparing evolutionary or genetic models that differ in complexity (for example, whether a more complex model allowing different mutation rates fits observed sequence data significantly better than a simpler, single-rate model) is a routine use of likelihood ratio testing in these fields.

Testing a specific parameter value in any MLE-fit model: the coin-flip example above generalizes directly — any time a model is fit via maximum likelihood estimation and there's a specific hypothesized value for one of its parameters, the likelihood ratio test provides a general-purpose way to test that hypothesis formally.

Common Mistakes

Comparing models that aren't actually nested. The test's chi-square reference distribution specifically relies on one model being a genuine restricted special case of the other — comparing two unrelated, non-nested models with this test produces a result with no valid statistical interpretation.

Forgetting to use the correct degrees of freedom. The degrees of freedom must equal the exact number of parameters fixed in going from the full model to the restricted one — getting this wrong (using the wrong chi-square reference distribution) directly produces an incorrect p-value.